-- Logs begin at Thu 2022-09-08 13:02:28 UTC, end at Thu 2022-09-08 14:13:01 UTC. -- Sep 08 13:37:54.431716 dv-cinder-820027-24-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 08 13:37:54.439826 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 08 13:37:54.442395 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 8 13:37:54 2022] *** Sep 08 13:37:54.442535 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 08 13:37:54.442655 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 08 13:37:54.442761 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: nodename: dv-cinder-820027-24-iscsi Sep 08 13:37:54.442862 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: machine: x86_64 Sep 08 13:37:54.442966 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: clock source: unix Sep 08 13:37:54.443066 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: pcre jit disabled Sep 08 13:37:54.443166 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: detected number of CPU cores: 4 Sep 08 13:37:54.443271 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: current working directory: / Sep 08 13:37:54.443372 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: detected binary path: /usr/bin/uwsgi-core Sep 08 13:37:54.443480 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: your processes number limit is 47680 Sep 08 13:37:54.443576 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: your memory page size is 4096 bytes Sep 08 13:37:54.443677 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: detected max file descriptor number: 2048 Sep 08 13:37:54.443783 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: lock engine: pthread robust mutexes Sep 08 13:37:54.444040 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: thunder lock: enabled Sep 08 13:37:54.444337 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 08 13:37:54.444440 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 08 13:37:54.483273 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: Python main interpreter initialized at 0x55f8a2fbe9c0 Sep 08 13:37:54.483273 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: python threads support enabled Sep 08 13:37:54.483273 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: your server socket listen backlog is limited to 100 connections Sep 08 13:37:54.483273 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: your mercy for graceful operations on workers is 90 seconds Sep 08 13:37:54.483620 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: mapped 403077 bytes (393 KB) for 2 cores Sep 08 13:37:54.483702 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: *** Operational MODE: preforking *** Sep 08 13:37:54.483949 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: *** uWSGI is running in multiple interpreter mode *** Sep 08 13:37:54.483949 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: spawned uWSGI master process (pid: 87903) Sep 08 13:37:54.484137 dv-cinder-820027-24-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 08 13:37:54.484870 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: spawned uWSGI worker 1 (pid: 87916, cores: 1) Sep 08 13:37:54.485032 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: spawned uWSGI worker 2 (pid: 87917, cores: 1) Sep 08 13:37:54.486847 dv-cinder-820027-24-iscsi devstack@keystone.service[87903]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 08 13:37:58.755586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:37:58.755586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: from cryptography.utils import int_from_bytes Sep 08 13:37:58.756385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:37:58.756385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: from cryptography.utils import int_from_bytes Sep 08 13:37:58.756531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:37:58.756531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: from cryptography.utils import int_from_bytes Sep 08 13:37:58.756531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:37:58.756531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: from cryptography.utils import int_from_bytes Sep 08 13:37:58.805362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 13:37:58.805537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 13:37:58.805723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] command line args: [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 13:37:58.806077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 13:37:58.806251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ================================================================================ {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 13:37:58.806501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] admin_token = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.806585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 13:37:58.806745 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] config_dir = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.806825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 13:37:58.806934 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.807020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] command line args: [] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 13:37:58.807098 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] config_source = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.807206 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 13:37:58.807287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] debug = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.807387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ================================================================================ {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 13:37:58.807471 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.807695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] admin_token = **** {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.807824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.807901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] config_dir = [] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.807969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.808072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.808217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.808335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] config_source = [] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.808443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] debug = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.808538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.808631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.808765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.808862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.808964 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_config_append = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.809065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.809160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.809160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_dir = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.809345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.809444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_file = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.809550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.809657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.809805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] list_limit = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.809921 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.810039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] log_config_append = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.810039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.810228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] log_dir = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.810350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] log_file = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.810427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.810585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.810709 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.810793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.810920 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.811002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.811075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.811187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.811305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.811454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.811550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.811627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.811743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.811842 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.811930 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.812077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.812152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.812305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.812382 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.812534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.812616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.812755 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.812877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.812988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.813065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.813233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.813477 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.813636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.813722 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] publish_errors = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.814358 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.814439 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.814660 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.814801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.55/identity {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.814888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.814970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] publish_errors = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.815058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.815160 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.815296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.815428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.815573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] standard_threads = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.815704 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.815837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.815969 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.816092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.816217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.816365 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.816497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] standard_threads = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.816628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] use_journal = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.816628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] use_json = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.816866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.816999 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] use_stderr = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.817190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.817318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] use_syslog = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.817443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.817567 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.817700 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] use_journal = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.817700 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] use_json = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.817967 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.818102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] use_stderr = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.818228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.818352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] use_syslog = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.818477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.818601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:37:58.818734 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.818865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.818991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.819123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.819249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.819373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.819506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.819637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.819760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.819895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.820019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.820144 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.820281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.820406 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.820537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.820657 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.820780 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.820905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.821041 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.821167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.821299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.821681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.821777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.821851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.821935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.822014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.822093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.822174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.822246 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.822327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.822408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.822481 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.822557 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.822634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.822717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.822790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.822937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.823017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.823095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.823172 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.823244 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.823322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.823400 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.823472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.823543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.823620 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.823691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.823772 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.823844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.823914 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.824034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.824110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.824186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.824308 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.824395 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.824541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.824619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.824779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.824865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.825028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.825276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.825362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.825507 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.825586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.825757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.825851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.826006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.826091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.826248 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.826325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.826483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.826569 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.826716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.826799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.826945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.827023 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.827175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.827261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.827410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.827488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.827638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.827714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.827869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.828024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.828099 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.828176 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.828315 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.828393 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.828549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.828635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.828778 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.828919 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.829021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.829124 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.829227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.829334 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.829467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.830155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.830270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.830366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.830507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.830602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.830743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.830821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.830976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.831058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.831209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.831296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.831442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.831520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.831671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.831762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.831905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.831983 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.832140 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.832216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.832368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.832446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.832597 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.832679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.832841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.832959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.833075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.833154 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.833304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.833384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.833537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.833631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.833791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.833917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.834073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.834162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.834295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.834412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.834489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.834609 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.834746 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.834826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.834980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.835061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.835215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.835302 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.835449 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.835528 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.835685 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.835764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.835917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.835995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.836152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.836229 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.836391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.836469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.836626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.836703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.836863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.836942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.837094 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.837174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.837331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.837410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.837562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.837639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.837818 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.837900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.838054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.838133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.838288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.838375 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.838524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.838604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.838761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.838839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.838996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.839074 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.839229 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.839309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.839466 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.839545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.839699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.839778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.839929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.840004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.840150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.840229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.840392 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.840480 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.840627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.840715 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.840862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.840941 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.841092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.841177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.841319 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.841405 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.841549 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.841627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.841803 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.841886 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.842034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.842124 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.842275 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.842352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.842510 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.842588 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.842758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.842844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.842996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.843075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.843225 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.external = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.843310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.843455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.843542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.843721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.843807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.843952 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] auth.external = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.844182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.844260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.844443 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.password = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.844529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.844676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] auth.token = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.844756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.844914 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] auth.password = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.845035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.845167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] auth.token = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.845247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.845428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.845576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.845656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.845833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.845942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.846081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.846177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.846327 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.846408 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.846556 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.846635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.846792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.846881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.847037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.847117 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] credential.caching = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.847288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.847368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.847532 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.847620 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.847772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.847852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.848013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.848091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.848245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.848332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.848479 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.848557 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.848724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.848803 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.848957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.849046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.849201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.849280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.849453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.849532 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.849712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.849844 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.849990 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.850068 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.850228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.850306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.850481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.850575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.850716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.850796 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.850949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.851035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] federation.caching = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.851183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.851270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.851422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.851507 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.851651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.851732 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.851882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.851972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.852116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.852194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.852349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.852428 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.852587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.852666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.852824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.852903 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.853058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.853136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.853299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.853379 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.853532 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.853611 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.853794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.caching = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.853875 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.854030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.854106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.854263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.854349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.854497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.854576 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.854731 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.854811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.854958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.855036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.855195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.855273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.855433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.855513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.855677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.855755 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.855906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.855985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.856134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.856221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.856361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.856438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.856590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.856669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.856832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.856912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.857073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.857151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.857316 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.857434 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.857556 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.857644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.857823 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.857905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.858071 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.858149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.858304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.858391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.858533 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.858612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.858763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.858847 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.858988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.859068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.859216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.859294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.859450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.859527 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.859683 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.859764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.859918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.859996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.860142 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.860219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.860383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.860462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.860623 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.860701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.860884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.860963 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.861119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.861198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.861358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.861435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.861586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.861673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.861840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.861921 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.862066 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.862144 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.862293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.862383 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.862524 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.862603 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.862757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.862835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.862987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.863072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.863220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.863331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.863562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.863642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.863813 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.863899 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.864044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.864122 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.864273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.864405 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.864533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.864668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.864798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.864927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.865051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.865129 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.865201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.865284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.865420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.865499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.865650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.865768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.866178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.866272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.866421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.866552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.866681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.866807 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.866945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.867080 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.867209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.867335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.867464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.867591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.867718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.867852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.867980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.868108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.868234 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.868361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.868490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.868634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.868761 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.868888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.869015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.869142 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.869269 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.869402 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.869531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.869658 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.869803 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.869926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.870050 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.870184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.870308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.870440 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.870566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.870688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.870811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.870942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.871065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.871189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.871312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.871435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.871559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.871690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.871813 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.871936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.872058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.872192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.872316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.872450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.872573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.872697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.872820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.872943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.873066 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.873197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.873321 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.873444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.873566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.873695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.874220 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.875552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.875799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.875925 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.876051 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.876179 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.876306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.876954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.877084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.877211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.877339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.877466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.877593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.877746 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.877883 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.878008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] resource.caching = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.878131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.878253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.878375 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.878508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.878631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.878754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.878876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.879000 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.879122 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.879257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.879381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.879503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.879636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.879759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.879958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.880085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.880208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] role.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.880330 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.880453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] role.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.880575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] role.caching = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.880705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.880828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] role.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.880950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.881073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.881195 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.881332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.881462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.881586 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.881709 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.881854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.881933 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882383 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.882998 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.883073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.883145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.883216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.883288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.883359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.883439 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.883513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.883585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.883656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.883728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.883799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.883879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.883953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.884072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.884149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.884221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.884301 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.884375 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.884447 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.884519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.884591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.884663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.884663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.884805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.884877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.884948 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.885019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.885136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.885221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.885295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.885295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.885424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.885496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.885567 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.885567 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.885704 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.885817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.885892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.885963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.886034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.886118 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.886193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.886265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.886337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.886411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.886483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] token.caching = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.886562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.886636 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.886707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.886779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.886850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.886921 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887289 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887439 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887727 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887903 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.887984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.888144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.888223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.888387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.888465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.888632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.888713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.888871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.888958 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.889123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.889211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.889367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.889508 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.connection = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.889602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.889684 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.889854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.889936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.connection = **** {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.890091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.890170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.890329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.890420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.890564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.890644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.890797 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.890876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.891030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.891118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.891263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.891344 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.891495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.891574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.891726 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.891813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.891986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.892069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.892757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.892886 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.893008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.893098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.893242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.893321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.893483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.893562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.893711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.893838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.893970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.894050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.894199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.894277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.894350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87916) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 13:37:58.894430 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.894665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:37:58.894835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87917) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 13:37:58.923674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:58.923822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:58.923912 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:58.924025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:58.924111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:58.924247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:58.932922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:58.934063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:58.962921 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:58.963146 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:58.964538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:58.964770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.017183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.017318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.066063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.066295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.067761 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.068003 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.224040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.224279 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.224369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.224504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.224619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.224726 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.224809 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.224942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.225023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.225159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.225250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.225371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.248776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.248915 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.249001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.249101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.249227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.249312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.249453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.249537 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.249673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.249777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.249917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.250001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.250131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.250230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.261910 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.262935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.275104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.275211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.275290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.275380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.275504 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.275697 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.275905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.275986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.276072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.276149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.276222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.276296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.276369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87917) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.276452 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87916) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:37:59.294957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.295166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:37:59.295515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.295719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:37:59.295997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.296264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.296570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.296835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.300123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.300337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:37:59.300689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.300889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:37:59.301166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.301434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.301699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.301992 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.309892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.310180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.310462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.310750 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.313221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.313506 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.313803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.314147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.318332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.321229 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.321800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.322095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.322387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.322676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.322944 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.323024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.323218 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.323498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.323776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.324055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.324327 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.325901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.326197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.326482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.326766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.327051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.327316 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.327590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.327867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.328139 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.328414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.328684 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.347019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.348495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.348786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.351257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.352720 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.353010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.353469 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.353787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.354081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.357659 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.357973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.358259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.360413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.361218 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.365098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.365401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.366958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.367265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.367566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.367869 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.368155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.368442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.368723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.369006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.371085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.371389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.371685 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.371978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.372263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.372547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.372827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.373100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.384932 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.385219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.389010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.389097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.389291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.392373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.392662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.393177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.396409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.396711 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.396894 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.400064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.400901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.403322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.403629 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.403920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.404030 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.404212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.404508 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.404804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.407301 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.408120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.408410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.408694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.408972 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.409252 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.421217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.421509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.421806 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.422078 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.425487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.425801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.426082 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.426349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.429561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.431137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.432669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.432954 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.433155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.434719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.435694 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.435984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.436248 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.436331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.436528 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.436610 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.436951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.439332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.439616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.439904 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.440191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.440599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.449446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.449759 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.450048 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.450339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.450729 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.451025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.451310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.452598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.452889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.453177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.453463 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.453870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.454165 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.454467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.466712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.469635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.469724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.473067 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.473158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.474550 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.475988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.476074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.476308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.476713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.477434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.478924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.479213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.479620 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.483110 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.484572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.486471 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.487931 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.488044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.488448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.488741 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.489024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.490835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.491236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.491528 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.491810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.496877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.497174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.497474 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.499554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.499859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.500145 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.504407 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87916) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.504698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.504987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.505273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.505564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.505865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.506145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.506427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.506702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.506997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.507076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87917) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:37:59.507284 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.507395 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.507570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.507669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.507852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.507943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.508129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.508210 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87916) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.508403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.508675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.508957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.509230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.509518 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.509833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.510101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.510379 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.510659 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87917) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:37:59.542593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f8a2fbe9c0 pid: 87916 (default app) Sep 08 13:37:59.544691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f8a2fbe9c0 pid: 87917 (default app) Sep 08 13:37:59.545861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73a46e-738b-4608-8edb-347622f2b584 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:59.546153 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73a46e-738b-4608-8edb-347622f2b584 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:59.546428 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73a46e-738b-4608-8edb-347622f2b584 None None] PATH_INFO: `/v3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:59.551194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1/1] 10.222.0.55 () {50 vars in 791 bytes} [Thu Sep 8 13:37:59 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:06.309714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149df855-07a6-4a73-885d-623310021c7c None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.310217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149df855-07a6-4a73-885d-623310021c7c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.310217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149df855-07a6-4a73-885d-623310021c7c None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.315001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1/2] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:06.322446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a33b5-3bc2-4393-959b-d4e385df7b67 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.322901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a33b5-3bc2-4393-959b-d4e385df7b67 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.323040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a33b5-3bc2-4393-959b-d4e385df7b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.394511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: INFO dbcounter [None req-4b2a33b5-3bc2-4393-959b-d4e385df7b67 None None] Registered counter for database keystone Sep 08 13:38:06.413885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG oslo_db.sqlalchemy.engines [None req-4b2a33b5-3bc2-4393-959b-d4e385df7b67 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87916) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:38:06.420024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writer thread running {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:38:06.599728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG passlib.handlers.bcrypt [None req-4b2a33b5-3bc2-4393-959b-d4e385df7b67 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87916) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 13:38:06.600041 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG passlib.handlers.bcrypt [None req-4b2a33b5-3bc2-4393-959b-d4e385df7b67 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87916) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 13:38:06.659115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4b2a33b5-3bc2-4393-959b-d4e385df7b67 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:06.660716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4b2a33b5-3bc2-4393-959b-d4e385df7b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:06.786978 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 2/3] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 08 13:38:07.257038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149df855-07a6-4a73-885d-623310021c7c None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.330511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: INFO dbcounter [None req-149df855-07a6-4a73-885d-623310021c7c None None] Registered counter for database keystone Sep 08 13:38:07.352108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG oslo_db.sqlalchemy.engines [None req-149df855-07a6-4a73-885d-623310021c7c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87917) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:38:07.358333 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writer thread running {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:38:07.477399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149df855-07a6-4a73-885d-623310021c7c None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.482254 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3033589-4ad3-46b8-a7d9-f1ddf0929dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.482914 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3033589-4ad3-46b8-a7d9-f1ddf0929dbe admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.483275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3033589-4ad3-46b8-a7d9-f1ddf0929dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.483553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3033589-4ad3-46b8-a7d9-f1ddf0929dbe admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.491847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3033589-4ad3-46b8-a7d9-f1ddf0929dbe admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.542867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f3033589-4ad3-46b8-a7d9-f1ddf0929dbe admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 13:38:07.543162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3033589-4ad3-46b8-a7d9-f1ddf0929dbe admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.552543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 2/4] 10.222.0.55 () {58 vars in 1324 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 13:38:07.558546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a33b5-3bc2-4393-959b-d4e385df7b67 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.575510 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a33b5-3bc2-4393-959b-d4e385df7b67 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.580414 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe9ef7a-a9da-4c82-a496-96f357540b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.581529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9ef7a-a9da-4c82-a496-96f357540b3d admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.581842 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9ef7a-a9da-4c82-a496-96f357540b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.582116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9ef7a-a9da-4c82-a496-96f357540b3d admin admin] PATH_INFO: `/v3/projects/ed368272c55a45f5a7959b08d8add92c` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.583704 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9ef7a-a9da-4c82-a496-96f357540b3d admin admin] RBAC: Authorizing `identity:get_project(project_id=ed368272c55a45f5a7959b08d8add92c)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.635853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9fe9ef7a-a9da-4c82-a496-96f357540b3d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 13:38:07.636194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe9ef7a-a9da-4c82-a496-96f357540b3d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.640727 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 3/5] 10.222.0.55 () {56 vars in 1207 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/projects/ed368272c55a45f5a7959b08d8add92c => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:09.754405 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35425358-31d0-49e3-887e-c443e762cd66 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.754949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35425358-31d0-49e3-887e-c443e762cd66 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.754949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35425358-31d0-49e3-887e-c443e762cd66 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.759065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 3/6] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:09.764480 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c61783-3c07-4ad9-99d1-591d6181ddd1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.764884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c61783-3c07-4ad9-99d1-591d6181ddd1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.765019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c61783-3c07-4ad9-99d1-591d6181ddd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.784748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-94c61783-3c07-4ad9-99d1-591d6181ddd1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:09.786280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-94c61783-3c07-4ad9-99d1-591d6181ddd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:09.794045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 4/7] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 08 13:38:10.061426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35425358-31d0-49e3-887e-c443e762cd66 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:10.073692 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35425358-31d0-49e3-887e-c443e762cd66 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:10.078013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2aac07-632a-4c10-955d-59d4c508f5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:10.078612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2aac07-632a-4c10-955d-59d4c508f5ef admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:10.078934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2aac07-632a-4c10-955d-59d4c508f5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:10.079209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2aac07-632a-4c10-955d-59d4c508f5ef admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:10.086819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2aac07-632a-4c10-955d-59d4c508f5ef admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:10.088595 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2aac07-632a-4c10-955d-59d4c508f5ef admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:10.097843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 4/8] 10.222.0.55 () {58 vars in 1324 bytes} [Thu Sep 8 13:38:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:38:10.103017 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c61783-3c07-4ad9-99d1-591d6181ddd1 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:10.112495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c61783-3c07-4ad9-99d1-591d6181ddd1 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:10.116764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe70cc2-e45e-451c-89b9-9b0ebfcaffb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:10.117388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe70cc2-e45e-451c-89b9-9b0ebfcaffb3 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:10.117667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe70cc2-e45e-451c-89b9-9b0ebfcaffb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:10.117970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe70cc2-e45e-451c-89b9-9b0ebfcaffb3 admin admin] PATH_INFO: `/v3/users/3709ff21e04749f7aae9073ca845e875` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:10.119279 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe70cc2-e45e-451c-89b9-9b0ebfcaffb3 admin admin] RBAC: Authorizing `identity:get_user(user_id=3709ff21e04749f7aae9073ca845e875)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:10.121100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe70cc2-e45e-451c-89b9-9b0ebfcaffb3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:10.122635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 5/9] 10.222.0.55 () {56 vars in 1198 bytes} [Thu Sep 8 13:38:10 2022] GET /identity/v3/users/3709ff21e04749f7aae9073ca845e875 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:12.334218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea62a40-ca1e-4769-ae98-380b0f0f6190 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.334704 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea62a40-ca1e-4769-ae98-380b0f0f6190 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.334704 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea62a40-ca1e-4769-ae98-380b0f0f6190 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.336014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 5/10] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:12.341197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6fbdc-02b6-4192-9785-546c419b3c9b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.341613 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6fbdc-02b6-4192-9785-546c419b3c9b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.341779 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6fbdc-02b6-4192-9785-546c419b3c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.353688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc494092-029d-476e-9665-15bd733b700f None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.354024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc494092-029d-476e-9665-15bd733b700f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.354305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc494092-029d-476e-9665-15bd733b700f None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.355617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 6/11] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:12.360826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a2e6fbdc-02b6-4192-9785-546c419b3c9b None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:12.361130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89887a4f-84b7-4deb-b09f-9772463f4bf1 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.361411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89887a4f-84b7-4deb-b09f-9772463f4bf1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.361678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89887a4f-84b7-4deb-b09f-9772463f4bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.362037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a2e6fbdc-02b6-4192-9785-546c419b3c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:12.402466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG passlib.handlers.bcrypt [None req-89887a4f-84b7-4deb-b09f-9772463f4bf1 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87917) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 13:38:12.402845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG passlib.handlers.bcrypt [None req-89887a4f-84b7-4deb-b09f-9772463f4bf1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87917) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 13:38:12.443232 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 6/12] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:12.452254 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-89887a4f-84b7-4deb-b09f-9772463f4bf1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:12.453918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-89887a4f-84b7-4deb-b09f-9772463f4bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:12.554771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 7/13] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:12.712316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6fbdc-02b6-4192-9785-546c419b3c9b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.736553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:12.815945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6fbdc-02b6-4192-9785-546c419b3c9b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:12.823045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89887a4f-84b7-4deb-b09f-9772463f4bf1 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.851019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da3463-4e98-4c37-849b-ff55097f0eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:12.851663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da3463-4e98-4c37-849b-ff55097f0eff None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.851942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da3463-4e98-4c37-849b-ff55097f0eff None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.852218 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da3463-4e98-4c37-849b-ff55097f0eff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.863337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da3463-4e98-4c37-849b-ff55097f0eff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-11da3463-4e98-4c37-849b-ff55097f0eff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:12.872299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:12.873212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:12.874077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:12.874077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:12.874077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:12.874077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:12.874077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:12.874077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:12.874077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:12.874077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:12.874077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:12.874077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:12.874077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:12.874077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:12.874077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 7/14] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:12.879756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da3463-4e98-4c37-849b-ff55097f0eff None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.924119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89887a4f-84b7-4deb-b09f-9772463f4bf1 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:12.926430 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:12.963696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0446e2-62de-41a0-a80a-35676ff0c1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:12.964488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0446e2-62de-41a0-a80a-35676ff0c1d0 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.964777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0446e2-62de-41a0-a80a-35676ff0c1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.965050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0446e2-62de-41a0-a80a-35676ff0c1d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.966850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:12.988235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da3463-4e98-4c37-849b-ff55097f0eff None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.022438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddceb111-3a13-42bf-8715-83c5ae7406b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.023052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddceb111-3a13-42bf-8715-83c5ae7406b5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.023329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddceb111-3a13-42bf-8715-83c5ae7406b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.023605 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddceb111-3a13-42bf-8715-83c5ae7406b5 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.024281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddceb111-3a13-42bf-8715-83c5ae7406b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:13.024584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddceb111-3a13-42bf-8715-83c5ae7406b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.026016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddceb111-3a13-42bf-8715-83c5ae7406b5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.034974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0446e2-62de-41a0-a80a-35676ff0c1d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.036484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0446e2-62de-41a0-a80a-35676ff0c1d0 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.037578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 8/15] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:13.043540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddceb111-3a13-42bf-8715-83c5ae7406b5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.136037 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 8/16] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:38:13.142138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0446e2-62de-41a0-a80a-35676ff0c1d0 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.143328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddceb111-3a13-42bf-8715-83c5ae7406b5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.144988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.177661 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43eea5-1949-4c29-9ba7-f9bd11cfdce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.178323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43eea5-1949-4c29-9ba7-f9bd11cfdce0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.178604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43eea5-1949-4c29-9ba7-f9bd11cfdce0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.178885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43eea5-1949-4c29-9ba7-f9bd11cfdce0 None admin] PATH_INFO: `/v3/users/3709ff21e04749f7aae9073ca845e875` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.180024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43eea5-1949-4c29-9ba7-f9bd11cfdce0 None admin] RBAC: Authorizing `identity:get_user(user_id=3709ff21e04749f7aae9073ca845e875)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.181472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43eea5-1949-4c29-9ba7-f9bd11cfdce0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.183052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 9/17] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/users/3709ff21e04749f7aae9073ca845e875 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:13.189188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43eea5-1949-4c29-9ba7-f9bd11cfdce0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.203111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.241530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0446e2-62de-41a0-a80a-35676ff0c1d0 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.274648 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbaae7-1f41-4b57-96d5-020153f9b33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.275497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbaae7-1f41-4b57-96d5-020153f9b33a None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.275779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbaae7-1f41-4b57-96d5-020153f9b33a None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.276053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbaae7-1f41-4b57-96d5-020153f9b33a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.286876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43eea5-1949-4c29-9ba7-f9bd11cfdce0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.295589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbaae7-1f41-4b57-96d5-020153f9b33a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.297965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbaae7-1f41-4b57-96d5-020153f9b33a None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-fabbaae7-1f41-4b57-96d5-020153f9b33a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:13.317162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:13.318260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:13.319075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:13.319075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:13.319075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:38:13.319075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 9/18] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:13.322907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375a862-6ca3-46b4-8b40-5a1bad5590b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.323628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375a862-6ca3-46b4-8b40-5a1bad5590b0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.323913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375a862-6ca3-46b4-8b40-5a1bad5590b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.324188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375a862-6ca3-46b4-8b40-5a1bad5590b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.324272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbaae7-1f41-4b57-96d5-020153f9b33a None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.325291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375a862-6ca3-46b4-8b40-5a1bad5590b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.326770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375a862-6ca3-46b4-8b40-5a1bad5590b0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.328296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 10/19] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:13.334018 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375a862-6ca3-46b4-8b40-5a1bad5590b0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.381925 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.414791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.428818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbaae7-1f41-4b57-96d5-020153f9b33a None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.434098 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375a862-6ca3-46b4-8b40-5a1bad5590b0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.462143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada8e4d-2fa6-4a51-a0e4-ad903dbe14a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.462747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada8e4d-2fa6-4a51-a0e4-ad903dbe14a5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.463023 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada8e4d-2fa6-4a51-a0e4-ad903dbe14a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.463292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada8e4d-2fa6-4a51-a0e4-ad903dbe14a5 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.463936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada8e4d-2fa6-4a51-a0e4-ad903dbe14a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:13.464232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada8e4d-2fa6-4a51-a0e4-ad903dbe14a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.465597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada8e4d-2fa6-4a51-a0e4-ad903dbe14a5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.467212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7b1fc-f3b0-4a25-bd09-d45e6dffa9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.467813 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7b1fc-f3b0-4a25-bd09-d45e6dffa9a3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.468086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7b1fc-f3b0-4a25-bd09-d45e6dffa9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.468352 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7b1fc-f3b0-4a25-bd09-d45e6dffa9a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.469060 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7b1fc-f3b0-4a25-bd09-d45e6dffa9a3 None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.domain.id=default, user.id=3709ff21e04749f7aae9073ca845e875) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:13.469395 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7b1fc-f3b0-4a25-bd09-d45e6dffa9a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.470824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7b1fc-f3b0-4a25-bd09-d45e6dffa9a3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.477329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 11/20] 10.222.0.55 () {56 vars in 1322 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/role_assignments?user.id=3709ff21e04749f7aae9073ca845e875&scope.domain.id=default&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:13.482610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 10/21] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:16.028090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bf6fe-57eb-42ad-9e78-d519203dea55 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.029115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bf6fe-57eb-42ad-9e78-d519203dea55 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.029618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793bf6fe-57eb-42ad-9e78-d519203dea55 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.031172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 12/22] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:16.037142 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eadf39-6a66-45a0-9d67-2707c4137b8c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.037996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eadf39-6a66-45a0-9d67-2707c4137b8c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.038498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eadf39-6a66-45a0-9d67-2707c4137b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.059865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-a7eadf39-6a66-45a0-9d67-2707c4137b8c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:16.061397 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a7eadf39-6a66-45a0-9d67-2707c4137b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:16.096564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b401403-2c03-4a37-a437-8205a3ff64f3 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.097454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b401403-2c03-4a37-a437-8205a3ff64f3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.097978 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b401403-2c03-4a37-a437-8205a3ff64f3 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.100474 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 13/23] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:16.106700 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194969d4-9086-4050-ab6c-d00f8ba049a1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.107245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194969d4-9086-4050-ab6c-d00f8ba049a1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.107880 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194969d4-9086-4050-ab6c-d00f8ba049a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.129220 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-194969d4-9086-4050-ab6c-d00f8ba049a1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:16.130798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-194969d4-9086-4050-ab6c-d00f8ba049a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:16.182637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 11/24] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:16.227798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 14/25] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:16.331541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3b6d2-af2d-4c09-b02c-9a9256b5da51 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.332192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3b6d2-af2d-4c09-b02c-9a9256b5da51 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.332693 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3b6d2-af2d-4c09-b02c-9a9256b5da51 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.334559 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 12/26] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:16.339819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5b777-5a1e-4482-8dea-3fdd4bb9fe7f None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.340365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5b777-5a1e-4482-8dea-3fdd4bb9fe7f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.340924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d5b777-5a1e-4482-8dea-3fdd4bb9fe7f None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.342369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 13/27] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:16.343450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a01778-b45c-450c-a412-aae6ab7818d3 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.343949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a01778-b45c-450c-a412-aae6ab7818d3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.344448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a01778-b45c-450c-a412-aae6ab7818d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.347497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e978ad4-dfeb-46a4-bf19-4b46a308be7d None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.348013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e978ad4-dfeb-46a4-bf19-4b46a308be7d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.348484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e978ad4-dfeb-46a4-bf19-4b46a308be7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.383883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4e978ad4-dfeb-46a4-bf19-4b46a308be7d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:16.385424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4e978ad4-dfeb-46a4-bf19-4b46a308be7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:16.388686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-21a01778-b45c-450c-a412-aae6ab7818d3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:16.390067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-21a01778-b45c-450c-a412-aae6ab7818d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:16.472424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:16.513152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 15/28] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:16.523312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 14/29] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:16.597203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a01778-b45c-450c-a412-aae6ab7818d3 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:16.603847 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:16.616733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e978ad4-dfeb-46a4-bf19-4b46a308be7d None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:16.769015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a01778-b45c-450c-a412-aae6ab7818d3 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.816535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f3549-2a56-421c-bd81-06bcf7e5096d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.817174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f3549-2a56-421c-bd81-06bcf7e5096d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.817448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f3549-2a56-421c-bd81-06bcf7e5096d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.817716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f3549-2a56-421c-bd81-06bcf7e5096d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.834176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e978ad4-dfeb-46a4-bf19-4b46a308be7d None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.852505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f3549-2a56-421c-bd81-06bcf7e5096d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-289f3549-2a56-421c-bd81-06bcf7e5096d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:16.857579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:16.858523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:16.859286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:16.859286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:16.859286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:16.859286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:16.859286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:16.859286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:16.859286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:16.859286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:16.859286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:16.859286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:16.859286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:16.859286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:16.862504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 16/30] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:16.863451 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f3549-2a56-421c-bd81-06bcf7e5096d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:16.944527 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e088c7-e4cb-4bff-a28c-1d5a1d0c0a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.945167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e088c7-e4cb-4bff-a28c-1d5a1d0c0a31 None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.945456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e088c7-e4cb-4bff-a28c-1d5a1d0c0a31 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.945754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e088c7-e4cb-4bff-a28c-1d5a1d0c0a31 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.946705 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e088c7-e4cb-4bff-a28c-1d5a1d0c0a31 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:16.948101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e088c7-e4cb-4bff-a28c-1d5a1d0c0a31 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:16.948406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:16.988562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f3549-2a56-421c-bd81-06bcf7e5096d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.024450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb5df3-469d-4ef3-b0b0-1e8fde0733a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.024984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb5df3-469d-4ef3-b0b0-1e8fde0733a9 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.025265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb5df3-469d-4ef3-b0b0-1e8fde0733a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.025540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb5df3-469d-4ef3-b0b0-1e8fde0733a9 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.026248 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb5df3-469d-4ef3-b0b0-1e8fde0733a9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.027629 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb5df3-469d-4ef3-b0b0-1e8fde0733a9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.097908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 15/31] 10.222.0.55 () {60 vars in 1131 bytes} [Thu Sep 8 13:38:16 2022] POST /identity/v3/roles => generated 219 bytes in 482 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:38:17.099611 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e088c7-e4cb-4bff-a28c-1d5a1d0c0a31 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.152770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:17.159513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 17/32] 10.222.0.55 () {60 vars in 1131 bytes} [Thu Sep 8 13:38:16 2022] POST /identity/v3/roles => generated 225 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:17.167728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb5df3-469d-4ef3-b0b0-1e8fde0733a9 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.297985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e088c7-e4cb-4bff-a28c-1d5a1d0c0a31 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.334379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb5df3-469d-4ef3-b0b0-1e8fde0733a9 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.355153 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485eb37-db6d-4144-b0f0-e1d84e386482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.355805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485eb37-db6d-4144-b0f0-e1d84e386482 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.356091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485eb37-db6d-4144-b0f0-e1d84e386482 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.356369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485eb37-db6d-4144-b0f0-e1d84e386482 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.357072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485eb37-db6d-4144-b0f0-e1d84e386482 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:17.357385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485eb37-db6d-4144-b0f0-e1d84e386482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.359000 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485eb37-db6d-4144-b0f0-e1d84e386482 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.370181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:17.378802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d8ccc-0a2b-4b00-9ca9-9a6b56abf45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.379908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d8ccc-0a2b-4b00-9ca9-9a6b56abf45f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.380408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d8ccc-0a2b-4b00-9ca9-9a6b56abf45f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.380929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d8ccc-0a2b-4b00-9ca9-9a6b56abf45f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.386075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 16/33] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:17.396091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485eb37-db6d-4144-b0f0-e1d84e386482 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.399903 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d8ccc-0a2b-4b00-9ca9-9a6b56abf45f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.402178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d8ccc-0a2b-4b00-9ca9-9a6b56abf45f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-326d8ccc-0a2b-4b00-9ca9-9a6b56abf45f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:17.425636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:17.426598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:17.427412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:17.427412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:17.427412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:17.432745 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 18/34] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:17.437895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d8ccc-0a2b-4b00-9ca9-9a6b56abf45f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.562976 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485eb37-db6d-4144-b0f0-e1d84e386482 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.566941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:17.604223 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d8ccc-0a2b-4b00-9ca9-9a6b56abf45f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.604732 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f63b8-b669-456e-8e43-ad2fe6ef84cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.605996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f63b8-b669-456e-8e43-ad2fe6ef84cb None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.606425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f63b8-b669-456e-8e43-ad2fe6ef84cb None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.606873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f63b8-b669-456e-8e43-ad2fe6ef84cb None admin] PATH_INFO: `/v3/users/3709ff21e04749f7aae9073ca845e875` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.608642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f63b8-b669-456e-8e43-ad2fe6ef84cb None admin] RBAC: Authorizing `identity:get_user(user_id=3709ff21e04749f7aae9073ca845e875)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.612882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41f63b8-b669-456e-8e43-ad2fe6ef84cb None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.613846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 17/35] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:17 2022] GET /identity/v3/users/3709ff21e04749f7aae9073ca845e875 => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:17.619427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f63b8-b669-456e-8e43-ad2fe6ef84cb None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.651636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d406bf-6af2-43cb-8835-a413d6fe9edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.652501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d406bf-6af2-43cb-8835-a413d6fe9edd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.652995 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d406bf-6af2-43cb-8835-a413d6fe9edd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.653452 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d406bf-6af2-43cb-8835-a413d6fe9edd None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.654409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d406bf-6af2-43cb-8835-a413d6fe9edd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:17.654901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d406bf-6af2-43cb-8835-a413d6fe9edd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.657074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d406bf-6af2-43cb-8835-a413d6fe9edd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.678036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 19/36] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:17.683702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d406bf-6af2-43cb-8835-a413d6fe9edd None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.727577 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:17.746336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f63b8-b669-456e-8e43-ad2fe6ef84cb None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.788815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d56a06-a027-4629-ba95-69240370e32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.789719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d56a06-a027-4629-ba95-69240370e32c None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.790256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d56a06-a027-4629-ba95-69240370e32c None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.790917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d56a06-a027-4629-ba95-69240370e32c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.793016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d56a06-a027-4629-ba95-69240370e32c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.794923 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d56a06-a027-4629-ba95-69240370e32c None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.796636 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 18/37] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:17.797636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d406bf-6af2-43cb-8835-a413d6fe9edd None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.803262 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d56a06-a027-4629-ba95-69240370e32c None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:17.846308 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:17.848373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb04851-4ef6-4dd6-8ba3-b0afb384120d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.849219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb04851-4ef6-4dd6-8ba3-b0afb384120d None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.849671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb04851-4ef6-4dd6-8ba3-b0afb384120d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.850109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb04851-4ef6-4dd6-8ba3-b0afb384120d None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.850940 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb04851-4ef6-4dd6-8ba3-b0afb384120d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.852593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb04851-4ef6-4dd6-8ba3-b0afb384120d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.887976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 20/38] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:38:17 2022] POST /identity/v3/projects => generated 301 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:38:17.932605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d56a06-a027-4629-ba95-69240370e32c None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:17.978431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb3f3e6-e798-4a4d-bc9e-0a364cd22cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:17.979393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f3e6-e798-4a4d-bc9e-0a364cd22cdb None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.979817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f3e6-e798-4a4d-bc9e-0a364cd22cdb None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.980264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb3f3e6-e798-4a4d-bc9e-0a364cd22cdb None admin] PATH_INFO: `/v3/domains/default/users/3709ff21e04749f7aae9073ca845e875/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.982504 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3f3e6-e798-4a4d-bc9e-0a364cd22cdb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3709ff21e04749f7aae9073ca845e875, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:17.984175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb3f3e6-e798-4a4d-bc9e-0a364cd22cdb None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:17.984813 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e2686-20cc-4659-a9ae-222665e988c7 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.985332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e2686-20cc-4659-a9ae-222665e988c7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.985864 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e2686-20cc-4659-a9ae-222665e988c7 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:17.987959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 21/39] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:17.994318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268481b-fcc3-4bcc-8619-4e77e500ac07 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:17.994887 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268481b-fcc3-4bcc-8619-4e77e500ac07 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:17.995344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268481b-fcc3-4bcc-8619-4e77e500ac07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:18.003769 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 19/40] 10.222.0.55 () {58 vars in 1361 bytes} [Thu Sep 8 13:38:17 2022] PUT /identity/v3/domains/default/users/3709ff21e04749f7aae9073ca845e875/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:18.019137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0268481b-fcc3-4bcc-8619-4e77e500ac07 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:18.020898 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0268481b-fcc3-4bcc-8619-4e77e500ac07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:18.067833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324b1bd-fcff-4ffa-bc76-6a615dc4f87f None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:18.068480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324b1bd-fcff-4ffa-bc76-6a615dc4f87f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:18.068981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324b1bd-fcff-4ffa-bc76-6a615dc4f87f None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:18.070735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 20/41] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:18.079006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291a8ae-f8bf-4246-9921-04ccd97ce8e6 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:18.079341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291a8ae-f8bf-4246-9921-04ccd97ce8e6 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:18.079611 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291a8ae-f8bf-4246-9921-04ccd97ce8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:18.105882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-2291a8ae-f8bf-4246-9921-04ccd97ce8e6 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:18.107436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2291a8ae-f8bf-4246-9921-04ccd97ce8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:18.199527 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 21/42] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:18.227376 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 22/43] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:18.466800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291a8ae-f8bf-4246-9921-04ccd97ce8e6 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:18.491255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268481b-fcc3-4bcc-8619-4e77e500ac07 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:18.493694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:18.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:18.581230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291a8ae-f8bf-4246-9921-04ccd97ce8e6 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:18.602173 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268481b-fcc3-4bcc-8619-4e77e500ac07 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:18.620439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0739b2-4323-426d-8090-a7680b125023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:18.621084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0739b2-4323-426d-8090-a7680b125023 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:18.621369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0739b2-4323-426d-8090-a7680b125023 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:18.621641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0739b2-4323-426d-8090-a7680b125023 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:18.623244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0739b2-4323-426d-8090-a7680b125023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:18.624701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0739b2-4323-426d-8090-a7680b125023 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:18.626504 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 22/44] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:18.631742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0739b2-4323-426d-8090-a7680b125023 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:18.636476 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462b28e-ba50-4def-b2e1-1adeee5e0ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:18.637091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462b28e-ba50-4def-b2e1-1adeee5e0ab3 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:18.637368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462b28e-ba50-4def-b2e1-1adeee5e0ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:18.637641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462b28e-ba50-4def-b2e1-1adeee5e0ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:18.638339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462b28e-ba50-4def-b2e1-1adeee5e0ab3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:18.639698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462b28e-ba50-4def-b2e1-1adeee5e0ab3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:18.657776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 23/45] 10.222.0.55 () {60 vars in 1131 bytes} [Thu Sep 8 13:38:18 2022] POST /identity/v3/roles => generated 223 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:38:18.734539 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:18.746405 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0739b2-4323-426d-8090-a7680b125023 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:18.785667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8250a-8277-4b62-9da3-550b8af81d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:18.786581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8250a-8277-4b62-9da3-550b8af81d8d None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:18.786865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8250a-8277-4b62-9da3-550b8af81d8d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:18.787137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8250a-8277-4b62-9da3-550b8af81d8d None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:18.787801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8250a-8277-4b62-9da3-550b8af81d8d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:18.789291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8250a-8277-4b62-9da3-550b8af81d8d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:18.829760 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 23/46] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:38:18 2022] POST /identity/v3/projects => generated 312 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:38:20.201494 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452f8dc-fa3f-4b55-ac68-34c7d2e4aba2 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.202024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452f8dc-fa3f-4b55-ac68-34c7d2e4aba2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.202024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452f8dc-fa3f-4b55-ac68-34c7d2e4aba2 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.203333 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 24/47] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:20.208703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c9f0e-1f8f-413b-9bb4-187a3d487f4f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.209126 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c9f0e-1f8f-413b-9bb4-187a3d487f4f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.209377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c9f0e-1f8f-413b-9bb4-187a3d487f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.227198 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-eb0c9f0e-1f8f-413b-9bb4-187a3d487f4f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:20.228366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-eb0c9f0e-1f8f-413b-9bb4-187a3d487f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:20.292408 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 24/48] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:20.564561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452f8dc-fa3f-4b55-ac68-34c7d2e4aba2 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:20.643738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:20.667496 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452f8dc-fa3f-4b55-ac68-34c7d2e4aba2 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:20.700095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72fa8b-caa8-4338-84a9-6c76d5693162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:20.700721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72fa8b-caa8-4338-84a9-6c76d5693162 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.701001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72fa8b-caa8-4338-84a9-6c76d5693162 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.701271 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72fa8b-caa8-4338-84a9-6c76d5693162 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.711646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72fa8b-caa8-4338-84a9-6c76d5693162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-0e72fa8b-caa8-4338-84a9-6c76d5693162 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:20.716442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:20.717316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:20.718103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:20.718103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:20.718103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:20.718103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:20.718103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:20.718103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:20.718103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:20.718103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:20.718103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:20.718103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:20.718103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:20.718103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:20.718103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 25/49] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:20.723793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c9f0e-1f8f-413b-9bb4-187a3d487f4f None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:20.795724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:20.821883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c9f0e-1f8f-413b-9bb4-187a3d487f4f None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:20.855010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ee42d-fe5d-4887-b1a7-c13a920aa02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:20.855617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ee42d-fe5d-4887-b1a7-c13a920aa02f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.855898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ee42d-fe5d-4887-b1a7-c13a920aa02f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.856169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ee42d-fe5d-4887-b1a7-c13a920aa02f None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.856834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ee42d-fe5d-4887-b1a7-c13a920aa02f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:20.857137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ee42d-fe5d-4887-b1a7-c13a920aa02f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:20.858556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ee42d-fe5d-4887-b1a7-c13a920aa02f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:20.867410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 25/50] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:20.873456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72fa8b-caa8-4338-84a9-6c76d5693162 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:20.974623 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72fa8b-caa8-4338-84a9-6c76d5693162 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:20.987537 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10569263-797e-42a4-9614-4b53376a4359 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.987828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10569263-797e-42a4-9614-4b53376a4359 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.988106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10569263-797e-42a4-9614-4b53376a4359 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.989443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 26/51] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:20.994305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9887222-1644-469c-a0d5-d6b3a0916b85 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.994586 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9887222-1644-469c-a0d5-d6b3a0916b85 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.995290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9887222-1644-469c-a0d5-d6b3a0916b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.008836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1701534-2bb9-464f-828f-d2f98bea69c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.009472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1701534-2bb9-464f-828f-d2f98bea69c0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.009783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1701534-2bb9-464f-828f-d2f98bea69c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.010064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1701534-2bb9-464f-828f-d2f98bea69c0 None admin] PATH_INFO: `/v3/users/3709ff21e04749f7aae9073ca845e875` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.011190 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1701534-2bb9-464f-828f-d2f98bea69c0 None admin] RBAC: Authorizing `identity:get_user(user_id=3709ff21e04749f7aae9073ca845e875)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.012646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1701534-2bb9-464f-828f-d2f98bea69c0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.013454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-d9887222-1644-469c-a0d5-d6b3a0916b85 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:21.014907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d9887222-1644-469c-a0d5-d6b3a0916b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:21.015010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 26/52] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:20 2022] GET /identity/v3/users/3709ff21e04749f7aae9073ca845e875 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:21.020192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1701534-2bb9-464f-828f-d2f98bea69c0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.029140 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:21.084324 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 27/53] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:21.118781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1701534-2bb9-464f-828f-d2f98bea69c0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.153235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c734b-3910-4b37-aace-8477b03340ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.153897 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c734b-3910-4b37-aace-8477b03340ec None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.154181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c734b-3910-4b37-aace-8477b03340ec None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.154458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c734b-3910-4b37-aace-8477b03340ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.155581 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c734b-3910-4b37-aace-8477b03340ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.157047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c734b-3910-4b37-aace-8477b03340ec None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.158632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 27/54] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:21.164533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9887222-1644-469c-a0d5-d6b3a0916b85 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.228805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:21.263541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9887222-1644-469c-a0d5-d6b3a0916b85 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.349580 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c734b-3910-4b37-aace-8477b03340ec None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.402161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498c11b-b3dc-4eeb-8a22-fda0393f7b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.402782 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498c11b-b3dc-4eeb-8a22-fda0393f7b38 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.403086 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498c11b-b3dc-4eeb-8a22-fda0393f7b38 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.403342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498c11b-b3dc-4eeb-8a22-fda0393f7b38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.404074 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498c11b-b3dc-4eeb-8a22-fda0393f7b38 None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.domain.id=default, user.id=3709ff21e04749f7aae9073ca845e875) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:21.404383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498c11b-b3dc-4eeb-8a22-fda0393f7b38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.405853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498c11b-b3dc-4eeb-8a22-fda0393f7b38 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.413648 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 28/55] 10.222.0.55 () {56 vars in 1322 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/role_assignments?user.id=3709ff21e04749f7aae9073ca845e875&scope.domain.id=default&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 530 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:21.425111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:21.453097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c734b-3910-4b37-aace-8477b03340ec None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.493558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f353b-af54-4f0e-99b7-7bd267fd2124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.494248 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f353b-af54-4f0e-99b7-7bd267fd2124 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.494536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f353b-af54-4f0e-99b7-7bd267fd2124 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.494811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f353b-af54-4f0e-99b7-7bd267fd2124 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.495993 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f353b-af54-4f0e-99b7-7bd267fd2124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.497468 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f353b-af54-4f0e-99b7-7bd267fd2124 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.499120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 28/56] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:21.504893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498c11b-b3dc-4eeb-8a22-fda0393f7b38 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.609687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498c11b-b3dc-4eeb-8a22-fda0393f7b38 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.638037 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:21.644950 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b63cbe-4635-4a0e-9988-10139397ab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.645563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b63cbe-4635-4a0e-9988-10139397ab8e None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b63cbe-4635-4a0e-9988-10139397ab8e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.646261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b63cbe-4635-4a0e-9988-10139397ab8e None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.646925 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b63cbe-4635-4a0e-9988-10139397ab8e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.648407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b63cbe-4635-4a0e-9988-10139397ab8e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.674083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 29/57] 10.222.0.55 () {60 vars in 1140 bytes} [Thu Sep 8 13:38:21 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:23.801661 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84641195-05f1-425b-a7d9-ad9a902dca9a None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.802207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84641195-05f1-425b-a7d9-ad9a902dca9a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.802287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84641195-05f1-425b-a7d9-ad9a902dca9a None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.803594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 29/58] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:23.808781 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33293e71-a2f2-4552-8286-f24f9eea8085 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.809201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33293e71-a2f2-4552-8286-f24f9eea8085 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.809337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33293e71-a2f2-4552-8286-f24f9eea8085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.827351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-33293e71-a2f2-4552-8286-f24f9eea8085 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:23.829041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-33293e71-a2f2-4552-8286-f24f9eea8085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:23.892703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 30/59] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:24.171795 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84641195-05f1-425b-a7d9-ad9a902dca9a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:24.277279 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84641195-05f1-425b-a7d9-ad9a902dca9a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:24.310135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f883f8-2dbd-41d4-90bc-c5cf3673f27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:24.310758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f883f8-2dbd-41d4-90bc-c5cf3673f27a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:24.311041 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f883f8-2dbd-41d4-90bc-c5cf3673f27a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:24.311318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f883f8-2dbd-41d4-90bc-c5cf3673f27a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:24.312453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f883f8-2dbd-41d4-90bc-c5cf3673f27a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:24.313921 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f883f8-2dbd-41d4-90bc-c5cf3673f27a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:24.315473 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 30/60] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:24.321231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33293e71-a2f2-4552-8286-f24f9eea8085 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:24.420476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33293e71-a2f2-4552-8286-f24f9eea8085 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:24.433811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:24.455065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74e119-f875-4ed2-9f54-0ee3f1ec9a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:24.455668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74e119-f875-4ed2-9f54-0ee3f1ec9a3d None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:24.455978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74e119-f875-4ed2-9f54-0ee3f1ec9a3d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:24.456253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74e119-f875-4ed2-9f54-0ee3f1ec9a3d None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:24.456918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74e119-f875-4ed2-9f54-0ee3f1ec9a3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:24.458412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74e119-f875-4ed2-9f54-0ee3f1ec9a3d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:24.483981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 31/61] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:24 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:38:27.609762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c058c3-a819-47d4-b2da-62b7f05fa07a None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.609762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c058c3-a819-47d4-b2da-62b7f05fa07a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.609762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c058c3-a819-47d4-b2da-62b7f05fa07a None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.610806 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 31/62] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:27.616389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cbe99-e163-4c37-a778-dc8762df1756 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.617151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cbe99-e163-4c37-a778-dc8762df1756 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.617596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cbe99-e163-4c37-a778-dc8762df1756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.641988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b27cbe99-e163-4c37-a778-dc8762df1756 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:27.643523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b27cbe99-e163-4c37-a778-dc8762df1756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:27.744006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 32/63] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:27.976781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f926394-4858-42e3-ab8a-5dfb4d506f62 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.977067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f926394-4858-42e3-ab8a-5dfb4d506f62 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.977509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f926394-4858-42e3-ab8a-5dfb4d506f62 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.979449 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 32/64] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:27.985667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbf059-ba72-4436-a42a-0b5388a7a9e4 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.986160 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbf059-ba72-4436-a42a-0b5388a7a9e4 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.986454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbf059-ba72-4436-a42a-0b5388a7a9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.009397 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b6fbf059-ba72-4436-a42a-0b5388a7a9e4 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:28.010699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b6fbf059-ba72-4436-a42a-0b5388a7a9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:28.115670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 33/65] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:28.123597 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f926394-4858-42e3-ab8a-5dfb4d506f62 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.136879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.138786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816b70a-645d-4b11-ba79-28f5da016ce3 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.139150 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816b70a-645d-4b11-ba79-28f5da016ce3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.139435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816b70a-645d-4b11-ba79-28f5da016ce3 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.140893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 34/66] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:28.147524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc380d-7769-4bfa-946b-097ed059c450 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.148144 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc380d-7769-4bfa-946b-097ed059c450 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.148619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc380d-7769-4bfa-946b-097ed059c450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.179254 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-ddbc380d-7769-4bfa-946b-097ed059c450 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:28.182085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ddbc380d-7769-4bfa-946b-097ed059c450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:28.232592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.286681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 35/67] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:28.288374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4d1be-0c46-46f0-bee0-72534835c4b0 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.288617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f926394-4858-42e3-ab8a-5dfb4d506f62 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.289093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4d1be-0c46-46f0-bee0-72534835c4b0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.289567 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4d1be-0c46-46f0-bee0-72534835c4b0 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.291277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 36/68] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:28.297288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd65341-8406-4364-af38-0b280b1228f3 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.297572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd65341-8406-4364-af38-0b280b1228f3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.297878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd65341-8406-4364-af38-0b280b1228f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.323146 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-ecd65341-8406-4364-af38-0b280b1228f3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:28.324343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ecd65341-8406-4364-af38-0b280b1228f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:28.359708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed77a8-5e19-4b19-81de-1a9a7a3a0f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.360350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed77a8-5e19-4b19-81de-1a9a7a3a0f8e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.360633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed77a8-5e19-4b19-81de-1a9a7a3a0f8e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.364970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed77a8-5e19-4b19-81de-1a9a7a3a0f8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.378348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed77a8-5e19-4b19-81de-1a9a7a3a0f8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-5fed77a8-5e19-4b19-81de-1a9a7a3a0f8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:28.383643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:28.384713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:28.385692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:28.385692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:28.385692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:28.385692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:28.385692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:28.385692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:28.385692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:28.385692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:28.385692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:28.385692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:28.385692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:28.385692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:28.388489 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 33/69] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:28.394552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed77a8-5e19-4b19-81de-1a9a7a3a0f8e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.458293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 37/70] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:28.459377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd65341-8406-4364-af38-0b280b1228f3 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.481253 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.528985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed77a8-5e19-4b19-81de-1a9a7a3a0f8e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.573455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.581975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd65341-8406-4364-af38-0b280b1228f3 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.594718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee3cb5-0ccd-4823-9824-1d4db5712194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.596139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee3cb5-0ccd-4823-9824-1d4db5712194 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.596822 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee3cb5-0ccd-4823-9824-1d4db5712194 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.597499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee3cb5-0ccd-4823-9824-1d4db5712194 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.598509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee3cb5-0ccd-4823-9824-1d4db5712194 None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:28.599067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee3cb5-0ccd-4823-9824-1d4db5712194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.600644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee3cb5-0ccd-4823-9824-1d4db5712194 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.610310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 34/71] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:28.611810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee3cb5-0ccd-4823-9824-1d4db5712194 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.621264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e988c-785f-4f14-ae17-62e70c4f5180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.621945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e988c-785f-4f14-ae17-62e70c4f5180 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.622231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e988c-785f-4f14-ae17-62e70c4f5180 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.622507 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e988c-785f-4f14-ae17-62e70c4f5180 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.635472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e988c-785f-4f14-ae17-62e70c4f5180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-cf4e988c-785f-4f14-ae17-62e70c4f5180 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:28.642474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:28.643488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 38/72] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:28.644466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e988c-785f-4f14-ae17-62e70c4f5180 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.724138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee3cb5-0ccd-4823-9824-1d4db5712194 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.754342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.758212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e988c-785f-4f14-ae17-62e70c4f5180 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.760022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5eccd-1442-482a-b8d1-168a281207a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.760665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5eccd-1442-482a-b8d1-168a281207a1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.760953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5eccd-1442-482a-b8d1-168a281207a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.761233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5eccd-1442-482a-b8d1-168a281207a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.772286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5eccd-1442-482a-b8d1-168a281207a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f3c5eccd-1442-482a-b8d1-168a281207a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:28.777668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:28.778624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:28.779487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:28.779487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:28.779487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:28.779487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:28.779487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:28.779487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:28.779487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:28.779487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:28.779487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:28.779487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:28.779487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:28.779487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:28.779487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 35/73] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:28.780285 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4833f1-b13e-4828-850b-2c382b092a22 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.780285 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4833f1-b13e-4828-850b-2c382b092a22 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.780458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4833f1-b13e-4828-850b-2c382b092a22 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.781723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 36/74] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:28.785979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4833f1-b13e-4828-850b-2c382b092a22 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.801050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdead75-6c18-45d4-91f8-5a41747714fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.801683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdead75-6c18-45d4-91f8-5a41747714fd None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.801998 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdead75-6c18-45d4-91f8-5a41747714fd None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.802276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdead75-6c18-45d4-91f8-5a41747714fd None admin] PATH_INFO: `/v3/users/9fe425c33e1e4fbeb81186f5585399d8` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.817502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdead75-6c18-45d4-91f8-5a41747714fd None admin] RBAC: Authorizing `identity:get_user(user_id=9fe425c33e1e4fbeb81186f5585399d8)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.819013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdead75-6c18-45d4-91f8-5a41747714fd None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.820869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 39/75] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/users/9fe425c33e1e4fbeb81186f5585399d8 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:28.821626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdead75-6c18-45d4-91f8-5a41747714fd None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.830466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.882889 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4833f1-b13e-4828-850b-2c382b092a22 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.916590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce460b9-bb02-4a5e-9b22-5b082f11517b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.917203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce460b9-bb02-4a5e-9b22-5b082f11517b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.917488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce460b9-bb02-4a5e-9b22-5b082f11517b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.917790 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce460b9-bb02-4a5e-9b22-5b082f11517b None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.918543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce460b9-bb02-4a5e-9b22-5b082f11517b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:28.918851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce460b9-bb02-4a5e-9b22-5b082f11517b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.920237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce460b9-bb02-4a5e-9b22-5b082f11517b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.924524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdead75-6c18-45d4-91f8-5a41747714fd None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.929932 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 37/76] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:28.930721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce460b9-bb02-4a5e-9b22-5b082f11517b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.962113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c078d40-0930-4f10-a7a5-16c58d921fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.962743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c078d40-0930-4f10-a7a5-16c58d921fc3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.963029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c078d40-0930-4f10-a7a5-16c58d921fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.963306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c078d40-0930-4f10-a7a5-16c58d921fc3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.973713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c078d40-0930-4f10-a7a5-16c58d921fc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-9c078d40-0930-4f10-a7a5-16c58d921fc3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:28.979063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:28.980031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:28.980798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:28.980798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:28.980798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:28.980798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:28.980798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:28.980798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:28.980798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:28.980798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:28.980798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:28.980798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:28.980798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:28.980798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:38:28.980798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 40/77] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:28.981456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2baf8-03ea-4819-a109-9f05e8dbd59f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.981456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2baf8-03ea-4819-a109-9f05e8dbd59f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.981964 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2baf8-03ea-4819-a109-9f05e8dbd59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.990042 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.999917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-fae2baf8-03ea-4819-a109-9f05e8dbd59f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:29.001125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-fae2baf8-03ea-4819-a109-9f05e8dbd59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:29.040523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce460b9-bb02-4a5e-9b22-5b082f11517b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.060300 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.068853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 41/78] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:29.069664 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2baf8-03ea-4819-a109-9f05e8dbd59f None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.083653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20312cbf-5815-4619-999c-267c982f1ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.084747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20312cbf-5815-4619-999c-267c982f1ecc None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.085226 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20312cbf-5815-4619-999c-267c982f1ecc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.086143 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20312cbf-5815-4619-999c-267c982f1ecc None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.087295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20312cbf-5815-4619-999c-267c982f1ecc None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:29.087772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20312cbf-5815-4619-999c-267c982f1ecc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.089945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20312cbf-5815-4619-999c-267c982f1ecc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.101631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 38/79] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:29.106822 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20312cbf-5815-4619-999c-267c982f1ecc None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.178270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2baf8-03ea-4819-a109-9f05e8dbd59f None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.210211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20312cbf-5815-4619-999c-267c982f1ecc None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.214200 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3377dfe-aef0-4a66-89c7-823209593623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.214821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3377dfe-aef0-4a66-89c7-823209593623 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.215104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3377dfe-aef0-4a66-89c7-823209593623 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.215379 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3377dfe-aef0-4a66-89c7-823209593623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.228308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.247892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ebb05-72b8-4d4c-9758-93ce0e035079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.248782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ebb05-72b8-4d4c-9758-93ce0e035079 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.249316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ebb05-72b8-4d4c-9758-93ce0e035079 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.249483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ebb05-72b8-4d4c-9758-93ce0e035079 None admin] PATH_INFO: `/v3/users/3709ff21e04749f7aae9073ca845e875` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.250830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ebb05-72b8-4d4c-9758-93ce0e035079 None admin] RBAC: Authorizing `identity:get_user(user_id=3709ff21e04749f7aae9073ca845e875)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.253182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ebb05-72b8-4d4c-9758-93ce0e035079 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.259359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 39/80] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/users/3709ff21e04749f7aae9073ca845e875 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:29.260044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ebb05-72b8-4d4c-9758-93ce0e035079 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.283563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.291078 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3377dfe-aef0-4a66-89c7-823209593623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.292567 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3377dfe-aef0-4a66-89c7-823209593623 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.369698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ebb05-72b8-4d4c-9758-93ce0e035079 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.392315 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 42/81] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:29.393157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3377dfe-aef0-4a66-89c7-823209593623 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.403645 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b0920-3392-4087-97b9-ee96eec9c880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.404310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b0920-3392-4087-97b9-ee96eec9c880 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.404596 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b0920-3392-4087-97b9-ee96eec9c880 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.404872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b0920-3392-4087-97b9-ee96eec9c880 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.405547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b0920-3392-4087-97b9-ee96eec9c880 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:29.405886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b0920-3392-4087-97b9-ee96eec9c880 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.407263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b0920-3392-4087-97b9-ee96eec9c880 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.420209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 40/82] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:29.420209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b0920-3392-4087-97b9-ee96eec9c880 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.463707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.486135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.492226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3377dfe-aef0-4a66-89c7-823209593623 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.517572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b0920-3392-4087-97b9-ee96eec9c880 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.527161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04eab44-85fe-4e86-a60c-7d3a17df2a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.527795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04eab44-85fe-4e86-a60c-7d3a17df2a82 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.528073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04eab44-85fe-4e86-a60c-7d3a17df2a82 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.528345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04eab44-85fe-4e86-a60c-7d3a17df2a82 None admin] PATH_INFO: `/v3/users/9fe425c33e1e4fbeb81186f5585399d8` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.529474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04eab44-85fe-4e86-a60c-7d3a17df2a82 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe425c33e1e4fbeb81186f5585399d8)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.531550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04eab44-85fe-4e86-a60c-7d3a17df2a82 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.533132 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 43/83] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/users/9fe425c33e1e4fbeb81186f5585399d8 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:29.534641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04eab44-85fe-4e86-a60c-7d3a17df2a82 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.551733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f86573-880f-4b11-a100-65f3be067a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.552337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f86573-880f-4b11-a100-65f3be067a5a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.552617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f86573-880f-4b11-a100-65f3be067a5a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.552893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f86573-880f-4b11-a100-65f3be067a5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.626334 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f86573-880f-4b11-a100-65f3be067a5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.627801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f86573-880f-4b11-a100-65f3be067a5a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.637945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04eab44-85fe-4e86-a60c-7d3a17df2a82 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.671149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.671296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ed73f-0a6a-4d50-8d6a-e169734e0a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.672686 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ed73f-0a6a-4d50-8d6a-e169734e0a32 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.672939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ed73f-0a6a-4d50-8d6a-e169734e0a32 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.673411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ed73f-0a6a-4d50-8d6a-e169734e0a32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.674678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ed73f-0a6a-4d50-8d6a-e169734e0a32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.676111 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ed73f-0a6a-4d50-8d6a-e169734e0a32 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.677751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 44/84] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:29.682956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ed73f-0a6a-4d50-8d6a-e169734e0a32 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.712535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.725295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 41/85] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:29.726260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f86573-880f-4b11-a100-65f3be067a5a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.783577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ed73f-0a6a-4d50-8d6a-e169734e0a32 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.816374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c168ee-82b7-4860-a41f-2d1b33d41763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.816994 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c168ee-82b7-4860-a41f-2d1b33d41763 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.817273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c168ee-82b7-4860-a41f-2d1b33d41763 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.817553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c168ee-82b7-4860-a41f-2d1b33d41763 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.818694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c168ee-82b7-4860-a41f-2d1b33d41763 None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.820122 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c168ee-82b7-4860-a41f-2d1b33d41763 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.822023 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 45/86] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:29.822786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c168ee-82b7-4860-a41f-2d1b33d41763 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.823039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f86573-880f-4b11-a100-65f3be067a5a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.860032 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3178b0-3d2a-4493-bb4c-e1fefc2be525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.860677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3178b0-3d2a-4493-bb4c-e1fefc2be525 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.860976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3178b0-3d2a-4493-bb4c-e1fefc2be525 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.861256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3178b0-3d2a-4493-bb4c-e1fefc2be525 None admin] PATH_INFO: `/v3/users/9fe425c33e1e4fbeb81186f5585399d8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.862416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3178b0-3d2a-4493-bb4c-e1fefc2be525 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe425c33e1e4fbeb81186f5585399d8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.863895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3178b0-3d2a-4493-bb4c-e1fefc2be525 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.865470 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 42/87] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/users/9fe425c33e1e4fbeb81186f5585399d8 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:29.866240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3178b0-3d2a-4493-bb4c-e1fefc2be525 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.883011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.929075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.932288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c168ee-82b7-4860-a41f-2d1b33d41763 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.966833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7c790-3e72-4e57-9efb-85b897bb5cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.967186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3178b0-3d2a-4493-bb4c-e1fefc2be525 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.967443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c790-3e72-4e57-9efb-85b897bb5cbc None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.967728 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c790-3e72-4e57-9efb-85b897bb5cbc None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.968008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7c790-3e72-4e57-9efb-85b897bb5cbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.000523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76efb39-484c-4d87-a67a-b2b0208f2909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.001128 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76efb39-484c-4d87-a67a-b2b0208f2909 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.001419 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76efb39-484c-4d87-a67a-b2b0208f2909 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.001689 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76efb39-484c-4d87-a67a-b2b0208f2909 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.002366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76efb39-484c-4d87-a67a-b2b0208f2909 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.003815 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76efb39-484c-4d87-a67a-b2b0208f2909 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.031782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 43/88] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:29 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:30.031782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76efb39-484c-4d87-a67a-b2b0208f2909 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.050156 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7c790-3e72-4e57-9efb-85b897bb5cbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.050156 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7c790-3e72-4e57-9efb-85b897bb5cbc None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.112982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.142005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.147136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 46/89] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:30.148290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7c790-3e72-4e57-9efb-85b897bb5cbc None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.155272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76efb39-484c-4d87-a67a-b2b0208f2909 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.193428 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f89d97-14a7-4e27-a35b-441e63331595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.194113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f89d97-14a7-4e27-a35b-441e63331595 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.194398 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f89d97-14a7-4e27-a35b-441e63331595 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.194680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f89d97-14a7-4e27-a35b-441e63331595 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.196127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f89d97-14a7-4e27-a35b-441e63331595 None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.197594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f89d97-14a7-4e27-a35b-441e63331595 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.199443 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 44/90] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:30.200057 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f89d97-14a7-4e27-a35b-441e63331595 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.263970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7c790-3e72-4e57-9efb-85b897bb5cbc None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.305708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e78e1-f8cb-44a9-909b-dafe43786453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.305708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e78e1-f8cb-44a9-909b-dafe43786453 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.305997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e78e1-f8cb-44a9-909b-dafe43786453 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.306237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e78e1-f8cb-44a9-909b-dafe43786453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.306842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e78e1-f8cb-44a9-909b-dafe43786453 None admin] RBAC: Adding query filter params (role.id=8abe8932c5974e7db6fb974c588a9d70, scope.project.id=32e961e117e24576acf2592d4ee51e41, user.id=9fe425c33e1e4fbeb81186f5585399d8) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:30.307156 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e78e1-f8cb-44a9-909b-dafe43786453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.308623 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e78e1-f8cb-44a9-909b-dafe43786453 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.315243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f89d97-14a7-4e27-a35b-441e63331595 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.315382 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 47/91] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/role_assignments?user.id=9fe425c33e1e4fbeb81186f5585399d8&scope.project.id=32e961e117e24576acf2592d4ee51e41&role.id=8abe8932c5974e7db6fb974c588a9d70 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:30.316151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e78e1-f8cb-44a9-909b-dafe43786453 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.370298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece080e3-2387-4ccb-aaba-820bc05b97cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.370999 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece080e3-2387-4ccb-aaba-820bc05b97cf None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.371332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece080e3-2387-4ccb-aaba-820bc05b97cf None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.371617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece080e3-2387-4ccb-aaba-820bc05b97cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.383679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.409620 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.462513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e78e1-f8cb-44a9-909b-dafe43786453 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.474089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece080e3-2387-4ccb-aaba-820bc05b97cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.476080 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece080e3-2387-4ccb-aaba-820bc05b97cf None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.506521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9b644-047d-4421-aed7-f2546861947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.507629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9b644-047d-4421-aed7-f2546861947a None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.508044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9b644-047d-4421-aed7-f2546861947a None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.508477 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9b644-047d-4421-aed7-f2546861947a None admin] PATH_INFO: `/v3/projects/1dde52f3fffc40b1b097a718ce6aab13` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.509689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9b644-047d-4421-aed7-f2546861947a None admin] RBAC: Authorizing `identity:get_project(project_id=1dde52f3fffc40b1b097a718ce6aab13)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.511233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9b644-047d-4421-aed7-f2546861947a None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.513401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 48/92] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/projects/1dde52f3fffc40b1b097a718ce6aab13 => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:30.514633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9b644-047d-4421-aed7-f2546861947a None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.623732 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 45/93] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:30.625159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece080e3-2387-4ccb-aaba-820bc05b97cf None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.653495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9b644-047d-4421-aed7-f2546861947a None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.672777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.686781 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.693877 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e997bf-9d7b-403e-99fb-8a8b5b6e39b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.694740 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e997bf-9d7b-403e-99fb-8a8b5b6e39b3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.695458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e997bf-9d7b-403e-99fb-8a8b5b6e39b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.696098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e997bf-9d7b-403e-99fb-8a8b5b6e39b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.696949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e997bf-9d7b-403e-99fb-8a8b5b6e39b3 None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=32e961e117e24576acf2592d4ee51e41, user.id=3709ff21e04749f7aae9073ca845e875) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:30.697383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e997bf-9d7b-403e-99fb-8a8b5b6e39b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.698982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e997bf-9d7b-403e-99fb-8a8b5b6e39b3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.705726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 49/94] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/role_assignments?user.id=3709ff21e04749f7aae9073ca845e875&scope.project.id=32e961e117e24576acf2592d4ee51e41&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:30.706573 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e997bf-9d7b-403e-99fb-8a8b5b6e39b3 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.783319 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece080e3-2387-4ccb-aaba-820bc05b97cf None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.841083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e997bf-9d7b-403e-99fb-8a8b5b6e39b3 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.880435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadac63e-cf17-4e00-808d-1c2388929a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.881012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadac63e-cf17-4e00-808d-1c2388929a71 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.881296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadac63e-cf17-4e00-808d-1c2388929a71 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.881576 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadac63e-cf17-4e00-808d-1c2388929a71 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.882798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadac63e-cf17-4e00-808d-1c2388929a71 None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.884274 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadac63e-cf17-4e00-808d-1c2388929a71 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.886180 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 50/95] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:30.894833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadac63e-cf17-4e00-808d-1c2388929a71 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.917507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658ac99-0c7f-4757-92e0-59674b854afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.918190 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658ac99-0c7f-4757-92e0-59674b854afb None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.918471 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658ac99-0c7f-4757-92e0-59674b854afb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.918743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658ac99-0c7f-4757-92e0-59674b854afb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.919505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658ac99-0c7f-4757-92e0-59674b854afb None admin] RBAC: Adding query filter params (role.id=8abe8932c5974e7db6fb974c588a9d70, scope.project.id=1dde52f3fffc40b1b097a718ce6aab13, user.id=9fe425c33e1e4fbeb81186f5585399d8) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:30.919819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658ac99-0c7f-4757-92e0-59674b854afb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.921303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658ac99-0c7f-4757-92e0-59674b854afb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.938971 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 46/96] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/role_assignments?user.id=9fe425c33e1e4fbeb81186f5585399d8&scope.project.id=1dde52f3fffc40b1b097a718ce6aab13&role.id=8abe8932c5974e7db6fb974c588a9d70 => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:31.017805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:31.073284 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadac63e-cf17-4e00-808d-1c2388929a71 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:31.125971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9698c1-8890-47d7-b75d-22e906c2c2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:31.126594 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9698c1-8890-47d7-b75d-22e906c2c2ca None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:31.126878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9698c1-8890-47d7-b75d-22e906c2c2ca None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:31.127148 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9698c1-8890-47d7-b75d-22e906c2c2ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:31.127893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9698c1-8890-47d7-b75d-22e906c2c2ca None admin] RBAC: Adding query filter params (role.id=7d0ac07c6114438da47dcfefeb1fe05b, scope.project.id=32e961e117e24576acf2592d4ee51e41, user.id=9fe425c33e1e4fbeb81186f5585399d8) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:31.128203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9698c1-8890-47d7-b75d-22e906c2c2ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:31.129650 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9698c1-8890-47d7-b75d-22e906c2c2ca None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:31.140274 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 51/97] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/role_assignments?user.id=9fe425c33e1e4fbeb81186f5585399d8&scope.project.id=32e961e117e24576acf2592d4ee51e41&role.id=7d0ac07c6114438da47dcfefeb1fe05b => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:33.200399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abcd4e-8d35-4622-8c23-c11ef2cf99f8 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.200899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abcd4e-8d35-4622-8c23-c11ef2cf99f8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.200899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abcd4e-8d35-4622-8c23-c11ef2cf99f8 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.202234 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 47/98] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:33.207811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc9380-4ab6-4f9b-b641-62ebaa33804c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.208318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc9380-4ab6-4f9b-b641-62ebaa33804c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.208404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc9380-4ab6-4f9b-b641-62ebaa33804c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.234778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-82dc9380-4ab6-4f9b-b641-62ebaa33804c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:33.235995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-82dc9380-4ab6-4f9b-b641-62ebaa33804c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:33.324165 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 52/99] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:33.368401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83691469-86ad-44aa-b0e4-28c14994ba0b None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.368699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83691469-86ad-44aa-b0e4-28c14994ba0b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.368978 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83691469-86ad-44aa-b0e4-28c14994ba0b None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.370348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 48/100] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:33.376117 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82d569-e91d-4ac8-892e-1e193836e711 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.376411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82d569-e91d-4ac8-892e-1e193836e711 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.376693 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82d569-e91d-4ac8-892e-1e193836e711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.404007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1d82d569-e91d-4ac8-892e-1e193836e711 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:33.406189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1d82d569-e91d-4ac8-892e-1e193836e711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:33.476855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:33.495039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 53/101] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:33.694175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75180b-d94d-49ee-8f61-69b28010ecd9 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.694579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75180b-d94d-49ee-8f61-69b28010ecd9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.694755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75180b-d94d-49ee-8f61-69b28010ecd9 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.696458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 49/102] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:33.711051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d098ed-2460-4f5e-93ff-e3b604a55cbb None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.711346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d098ed-2460-4f5e-93ff-e3b604a55cbb None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.711635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d098ed-2460-4f5e-93ff-e3b604a55cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.738782 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-85d098ed-2460-4f5e-93ff-e3b604a55cbb None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:33.739903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-85d098ed-2460-4f5e-93ff-e3b604a55cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:33.828926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75180b-d94d-49ee-8f61-69b28010ecd9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.829211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 54/103] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:33.888120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc921a86-168e-4bd3-b78d-31583c44dd1b None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.888421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc921a86-168e-4bd3-b78d-31583c44dd1b None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.888680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc921a86-168e-4bd3-b78d-31583c44dd1b None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.890855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 55/104] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:33.902229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0747f2-d750-4291-8d72-62612269f085 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.902869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0747f2-d750-4291-8d72-62612269f085 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.909646 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0747f2-d750-4291-8d72-62612269f085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.931527 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-bc0747f2-d750-4291-8d72-62612269f085 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:33.933056 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-bc0747f2-d750-4291-8d72-62612269f085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:33.995088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75180b-d94d-49ee-8f61-69b28010ecd9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:33.999029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.058198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b9b54-888b-4060-9c27-9e0e5097782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.060922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b9b54-888b-4060-9c27-9e0e5097782e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.061214 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b9b54-888b-4060-9c27-9e0e5097782e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.061492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6b9b54-888b-4060-9c27-9e0e5097782e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.067377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 56/105] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:34.069518 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0747f2-d750-4291-8d72-62612269f085 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.086572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6b9b54-888b-4060-9c27-9e0e5097782e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-8f6b9b54-888b-4060-9c27-9e0e5097782e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:34.096039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:34.096899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:34.097663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:34.097663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:34.097663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:34.097663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:34.097663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:34.097663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:34.097663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:34.097663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:34.097663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:34.097663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:34.097663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:38:34.097663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:34.102008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 50/106] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:34.127728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b9b54-888b-4060-9c27-9e0e5097782e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.170424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.241090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0747f2-d750-4291-8d72-62612269f085 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.265551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6b9b54-888b-4060-9c27-9e0e5097782e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.288636 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89177cf2-6dea-4cbe-95c8-90a5db42a684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.289286 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89177cf2-6dea-4cbe-95c8-90a5db42a684 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.289851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89177cf2-6dea-4cbe-95c8-90a5db42a684 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.290141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89177cf2-6dea-4cbe-95c8-90a5db42a684 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.291686 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89177cf2-6dea-4cbe-95c8-90a5db42a684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.293685 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89177cf2-6dea-4cbe-95c8-90a5db42a684 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.296051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 57/107] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:34.298475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff1eec-ab2e-4ad6-bd87-f688b5214a10 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.298784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff1eec-ab2e-4ad6-bd87-f688b5214a10 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.299064 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff1eec-ab2e-4ad6-bd87-f688b5214a10 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.300728 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 58/108] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:34.301423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff1eec-ab2e-4ad6-bd87-f688b5214a10 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.304273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e619ec9-ebc4-4469-8ff2-915145fefd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.305155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619ec9-ebc4-4469-8ff2-915145fefd94 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.305634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619ec9-ebc4-4469-8ff2-915145fefd94 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.306164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619ec9-ebc4-4469-8ff2-915145fefd94 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.307097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e619ec9-ebc4-4469-8ff2-915145fefd94 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:34.307614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e619ec9-ebc4-4469-8ff2-915145fefd94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.309233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e619ec9-ebc4-4469-8ff2-915145fefd94 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.319300 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 51/109] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:34.320332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e619ec9-ebc4-4469-8ff2-915145fefd94 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.351856 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.429473 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e619ec9-ebc4-4469-8ff2-915145fefd94 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.435234 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beff1eec-ab2e-4ad6-bd87-f688b5214a10 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.453224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.465256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437a609-beb9-48d2-b20e-7206c63a4722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.466188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437a609-beb9-48d2-b20e-7206c63a4722 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.466650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437a609-beb9-48d2-b20e-7206c63a4722 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.467094 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437a609-beb9-48d2-b20e-7206c63a4722 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.467925 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437a609-beb9-48d2-b20e-7206c63a4722 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.470049 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437a609-beb9-48d2-b20e-7206c63a4722 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.473084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579b012-4ebc-4ef3-8cb0-8def78937e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.473807 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579b012-4ebc-4ef3-8cb0-8def78937e69 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.474095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579b012-4ebc-4ef3-8cb0-8def78937e69 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.474371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579b012-4ebc-4ef3-8cb0-8def78937e69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.487417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579b012-4ebc-4ef3-8cb0-8def78937e69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-9579b012-4ebc-4ef3-8cb0-8def78937e69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:34.492810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:34.493981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:34.494775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:34.494775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:34.494775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:34.494775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:34.494775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:34.494775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:34.494775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:34.494775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:34.494775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:34.494775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:34.494775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:34.494775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:38:34.494775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 59/110] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:34.495425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5015db-fbb3-4541-a5f6-9e062455e70a None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.495528 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5015db-fbb3-4541-a5f6-9e062455e70a None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.495802 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5015db-fbb3-4541-a5f6-9e062455e70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.503344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 52/111] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:38:34 2022] POST /identity/v3/projects => generated 302 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:34.504506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437a609-beb9-48d2-b20e-7206c63a4722 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.527711 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1d5015db-fbb3-4541-a5f6-9e062455e70a None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:34.530233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1d5015db-fbb3-4541-a5f6-9e062455e70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:34.606575 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.624894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 60/112] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:34.626107 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5015db-fbb3-4541-a5f6-9e062455e70a None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.650662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437a609-beb9-48d2-b20e-7206c63a4722 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.701105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b78af-3183-44c1-9a68-752e380fd3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.701793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b78af-3183-44c1-9a68-752e380fd3f0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.702090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b78af-3183-44c1-9a68-752e380fd3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.702374 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b78af-3183-44c1-9a68-752e380fd3f0 None admin] PATH_INFO: `/v3/users/e58d91e284ee49b7b6cf95625c3c9dc2` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.719167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b78af-3183-44c1-9a68-752e380fd3f0 None admin] RBAC: Authorizing `identity:get_user(user_id=e58d91e284ee49b7b6cf95625c3c9dc2)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.720654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b78af-3183-44c1-9a68-752e380fd3f0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.722520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 53/113] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/users/e58d91e284ee49b7b6cf95625c3c9dc2 => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:34.723212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b78af-3183-44c1-9a68-752e380fd3f0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.786891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5015db-fbb3-4541-a5f6-9e062455e70a None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.795363 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.838116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8f978-9ea0-4658-a1e9-47e0a95233b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.839122 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8f978-9ea0-4658-a1e9-47e0a95233b4 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.839598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8f978-9ea0-4658-a1e9-47e0a95233b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.840242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8f978-9ea0-4658-a1e9-47e0a95233b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.848370 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b78af-3183-44c1-9a68-752e380fd3f0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.854486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c8f978-9ea0-4658-a1e9-47e0a95233b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-88c8f978-9ea0-4658-a1e9-47e0a95233b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:34.860883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:34.861916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:34.862859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:34.862859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:34.862859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:34.862859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:34.862859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:34.862859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:34.862859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:34.862859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:34.862859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:34.862859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:34.862859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:34.862859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:38:34.865024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 61/114] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:34.866692 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6477e-1493-423f-bc50-d39957feff65 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.867116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6477e-1493-423f-bc50-d39957feff65 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.867997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6477e-1493-423f-bc50-d39957feff65 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.871868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 62/115] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:34.872815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6477e-1493-423f-bc50-d39957feff65 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.902410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac8a03-49b3-48f4-a6c1-d5a67431e87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.903278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac8a03-49b3-48f4-a6c1-d5a67431e87c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.903713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac8a03-49b3-48f4-a6c1-d5a67431e87c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.904172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac8a03-49b3-48f4-a6c1-d5a67431e87c None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.904972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac8a03-49b3-48f4-a6c1-d5a67431e87c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:34.905500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac8a03-49b3-48f4-a6c1-d5a67431e87c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.907081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac8a03-49b3-48f4-a6c1-d5a67431e87c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.908630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.917325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 54/116] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:34.918165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac8a03-49b3-48f4-a6c1-d5a67431e87c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.994990 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6477e-1493-423f-bc50-d39957feff65 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.022265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac8a03-49b3-48f4-a6c1-d5a67431e87c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.030179 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c345dae-9ada-471b-9b0a-93b7ac2cacb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.030987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c345dae-9ada-471b-9b0a-93b7ac2cacb7 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.031756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c345dae-9ada-471b-9b0a-93b7ac2cacb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.032036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c345dae-9ada-471b-9b0a-93b7ac2cacb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.057957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c73575-e95b-41b7-8c5b-64dd3a3a5960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.058641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c73575-e95b-41b7-8c5b-64dd3a3a5960 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.058933 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c73575-e95b-41b7-8c5b-64dd3a3a5960 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.059211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c73575-e95b-41b7-8c5b-64dd3a3a5960 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.059882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c73575-e95b-41b7-8c5b-64dd3a3a5960 None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:35.060189 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c73575-e95b-41b7-8c5b-64dd3a3a5960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.061576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c73575-e95b-41b7-8c5b-64dd3a3a5960 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.067587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.070856 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 55/117] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:35.072271 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032f527-5011-45f6-95c0-b709751b20c9 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.072566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032f527-5011-45f6-95c0-b709751b20c9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.072846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3032f527-5011-45f6-95c0-b709751b20c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.091568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-3032f527-5011-45f6-95c0-b709751b20c9 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:35.092878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3032f527-5011-45f6-95c0-b709751b20c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:35.112409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c345dae-9ada-471b-9b0a-93b7ac2cacb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.114229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c345dae-9ada-471b-9b0a-93b7ac2cacb7 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.152413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.167957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 56/118] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:35.168948 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032f527-5011-45f6-95c0-b709751b20c9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.232306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 63/119] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:35.233290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c345dae-9ada-471b-9b0a-93b7ac2cacb7 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.294571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3032f527-5011-45f6-95c0-b709751b20c9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.311115 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.342819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd488e27-35f4-42ea-9667-a63ee9aa7c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.344110 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd488e27-35f4-42ea-9667-a63ee9aa7c41 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.344611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd488e27-35f4-42ea-9667-a63ee9aa7c41 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.345473 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd488e27-35f4-42ea-9667-a63ee9aa7c41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.354686 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c345dae-9ada-471b-9b0a-93b7ac2cacb7 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.360467 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd488e27-35f4-42ea-9667-a63ee9aa7c41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-cd488e27-35f4-42ea-9667-a63ee9aa7c41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:35.366719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:35.367576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:35.368344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:35.368344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:35.368344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:35.368344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:35.368344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:35.368344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:35.368344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:35.368344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:35.368344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:35.368344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:35.368344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:35.368344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:35.371304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 57/120] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:35.372300 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd488e27-35f4-42ea-9667-a63ee9aa7c41 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.398012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab20bee-19f8-4363-913a-7a75da0506ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.398983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab20bee-19f8-4363-913a-7a75da0506ae None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.399460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab20bee-19f8-4363-913a-7a75da0506ae None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.399925 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab20bee-19f8-4363-913a-7a75da0506ae None admin] PATH_INFO: `/v3/users/3709ff21e04749f7aae9073ca845e875` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.401300 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab20bee-19f8-4363-913a-7a75da0506ae None admin] RBAC: Authorizing `identity:get_user(user_id=3709ff21e04749f7aae9073ca845e875)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.403007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab20bee-19f8-4363-913a-7a75da0506ae None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.405305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 64/121] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/users/3709ff21e04749f7aae9073ca845e875 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:35.406812 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab20bee-19f8-4363-913a-7a75da0506ae None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.438126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.484638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd488e27-35f4-42ea-9667-a63ee9aa7c41 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.521366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe9dd0-8a71-4678-b12d-88c6c54e0119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.522226 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe9dd0-8a71-4678-b12d-88c6c54e0119 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.522402 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab20bee-19f8-4363-913a-7a75da0506ae None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.522571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe9dd0-8a71-4678-b12d-88c6c54e0119 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.522825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe9dd0-8a71-4678-b12d-88c6c54e0119 None admin] PATH_INFO: `/v3/users/9fe425c33e1e4fbeb81186f5585399d8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.524040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe9dd0-8a71-4678-b12d-88c6c54e0119 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe425c33e1e4fbeb81186f5585399d8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.525821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe9dd0-8a71-4678-b12d-88c6c54e0119 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.527254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 58/122] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/users/9fe425c33e1e4fbeb81186f5585399d8 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:35.531156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe9dd0-8a71-4678-b12d-88c6c54e0119 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.564085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96490053-30fa-4727-b8d0-d0b1e8c98b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.564650 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96490053-30fa-4727-b8d0-d0b1e8c98b9d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.564934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96490053-30fa-4727-b8d0-d0b1e8c98b9d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.565269 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96490053-30fa-4727-b8d0-d0b1e8c98b9d None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.566909 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96490053-30fa-4727-b8d0-d0b1e8c98b9d None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.568655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96490053-30fa-4727-b8d0-d0b1e8c98b9d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.570944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 65/123] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:35.571401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96490053-30fa-4727-b8d0-d0b1e8c98b9d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.574508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.643692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe9dd0-8a71-4678-b12d-88c6c54e0119 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.660801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.679220 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463df2cb-91f0-4300-b047-288b564359ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.679830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463df2cb-91f0-4300-b047-288b564359ff None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.680112 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463df2cb-91f0-4300-b047-288b564359ff None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.680387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463df2cb-91f0-4300-b047-288b564359ff None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.681049 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463df2cb-91f0-4300-b047-288b564359ff None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:35.681356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463df2cb-91f0-4300-b047-288b564359ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.682776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463df2cb-91f0-4300-b047-288b564359ff None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.687027 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96490053-30fa-4727-b8d0-d0b1e8c98b9d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.690997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 59/124] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:35.696485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463df2cb-91f0-4300-b047-288b564359ff None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.723321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd47ef-b3c1-4c4e-9af9-a96843c91d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.724241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd47ef-b3c1-4c4e-9af9-a96843c91d0f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.724696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd47ef-b3c1-4c4e-9af9-a96843c91d0f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.725139 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd47ef-b3c1-4c4e-9af9-a96843c91d0f None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.726397 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd47ef-b3c1-4c4e-9af9-a96843c91d0f None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.728044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd47ef-b3c1-4c4e-9af9-a96843c91d0f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.735513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 66/125] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:35.736529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd47ef-b3c1-4c4e-9af9-a96843c91d0f None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.797543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463df2cb-91f0-4300-b047-288b564359ff None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.811945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.832491 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cff59f-b4ab-4043-98a2-f62e080d2232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.833055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cff59f-b4ab-4043-98a2-f62e080d2232 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.833720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cff59f-b4ab-4043-98a2-f62e080d2232 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.837425 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cff59f-b4ab-4043-98a2-f62e080d2232 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.846845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd47ef-b3c1-4c4e-9af9-a96843c91d0f None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.854531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cff59f-b4ab-4043-98a2-f62e080d2232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f5cff59f-b4ab-4043-98a2-f62e080d2232 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:35.862341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:35.863369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:35.864318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:35.864318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:35.864318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:35.864318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:35.864318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:35.864318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:35.864318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:35.864318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:35.864318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:35.864318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:35.864318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:35.864318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:35.866682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 60/126] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:35.867603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cff59f-b4ab-4043-98a2-f62e080d2232 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.888377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80564f26-7c8c-4fc9-97bb-282090f23b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.889328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80564f26-7c8c-4fc9-97bb-282090f23b6c None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.889608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80564f26-7c8c-4fc9-97bb-282090f23b6c None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.890403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80564f26-7c8c-4fc9-97bb-282090f23b6c None admin] PATH_INFO: `/v3/projects/1dde52f3fffc40b1b097a718ce6aab13` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.891531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80564f26-7c8c-4fc9-97bb-282090f23b6c None admin] RBAC: Authorizing `identity:get_project(project_id=1dde52f3fffc40b1b097a718ce6aab13)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.893017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80564f26-7c8c-4fc9-97bb-282090f23b6c None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.899216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 67/127] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/projects/1dde52f3fffc40b1b097a718ce6aab13 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:35.899216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80564f26-7c8c-4fc9-97bb-282090f23b6c None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.917207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.982291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cff59f-b4ab-4043-98a2-f62e080d2232 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.011171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80564f26-7c8c-4fc9-97bb-282090f23b6c None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.017064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4c5f6-d32e-4ad3-b9ee-b32f6f85f65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.018011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4c5f6-d32e-4ad3-b9ee-b32f6f85f65f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.018625 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4c5f6-d32e-4ad3-b9ee-b32f6f85f65f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.018966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4c5f6-d32e-4ad3-b9ee-b32f6f85f65f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.019701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4c5f6-d32e-4ad3-b9ee-b32f6f85f65f None admin] RBAC: Adding query filter params (role.id=eeefde0ca97549ed9cd98300215a29ad, scope.project.id=32e961e117e24576acf2592d4ee51e41, user.id=e58d91e284ee49b7b6cf95625c3c9dc2) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:36.020011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4c5f6-d32e-4ad3-b9ee-b32f6f85f65f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.021584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4c5f6-d32e-4ad3-b9ee-b32f6f85f65f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.037768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 61/128] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/role_assignments?user.id=e58d91e284ee49b7b6cf95625c3c9dc2&scope.project.id=32e961e117e24576acf2592d4ee51e41&role.id=eeefde0ca97549ed9cd98300215a29ad => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:36.037768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4c5f6-d32e-4ad3-b9ee-b32f6f85f65f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.057881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124aedce-f0ee-403b-a968-ef47fe57abe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.058544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124aedce-f0ee-403b-a968-ef47fe57abe3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.058896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124aedce-f0ee-403b-a968-ef47fe57abe3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.059241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124aedce-f0ee-403b-a968-ef47fe57abe3 None admin] PATH_INFO: `/v3/users/9fe425c33e1e4fbeb81186f5585399d8` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.060500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124aedce-f0ee-403b-a968-ef47fe57abe3 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe425c33e1e4fbeb81186f5585399d8)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.062172 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124aedce-f0ee-403b-a968-ef47fe57abe3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.068740 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 68/129] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/users/9fe425c33e1e4fbeb81186f5585399d8 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:36.070099 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124aedce-f0ee-403b-a968-ef47fe57abe3 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.081291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.177249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4c5f6-d32e-4ad3-b9ee-b32f6f85f65f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.179683 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.190948 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124aedce-f0ee-403b-a968-ef47fe57abe3 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.230943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8105841-756b-4c49-bb56-a7708e6fa4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.231536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37666474-3dbe-4e85-afcd-1d9fcef8d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.231896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8105841-756b-4c49-bb56-a7708e6fa4bb None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.232151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37666474-3dbe-4e85-afcd-1d9fcef8d241 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.232245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8105841-756b-4c49-bb56-a7708e6fa4bb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.232433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37666474-3dbe-4e85-afcd-1d9fcef8d241 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.232547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8105841-756b-4c49-bb56-a7708e6fa4bb None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41/users/3709ff21e04749f7aae9073ca845e875/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.232710 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37666474-3dbe-4e85-afcd-1d9fcef8d241 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.234536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37666474-3dbe-4e85-afcd-1d9fcef8d241 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:36.234864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37666474-3dbe-4e85-afcd-1d9fcef8d241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.236434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37666474-3dbe-4e85-afcd-1d9fcef8d241 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.240558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8105841-756b-4c49-bb56-a7708e6fa4bb None admin] RBAC: Authorizing `identity:create_grant(project_id=32e961e117e24576acf2592d4ee51e41, user_id=3709ff21e04749f7aae9073ca845e875, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.241411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8105841-756b-4c49-bb56-a7708e6fa4bb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.251698 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 69/130] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:36.252594 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37666474-3dbe-4e85-afcd-1d9fcef8d241 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.260540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 62/131] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:36 2022] PUT /identity/v3/projects/32e961e117e24576acf2592d4ee51e41/users/3709ff21e04749f7aae9073ca845e875/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:36.267205 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8105841-756b-4c49-bb56-a7708e6fa4bb None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.330512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.370617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37666474-3dbe-4e85-afcd-1d9fcef8d241 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.416194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0580a-a024-4705-a47c-43f594cf4d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.416194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0580a-a024-4705-a47c-43f594cf4d20 None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.416194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0580a-a024-4705-a47c-43f594cf4d20 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.416194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0580a-a024-4705-a47c-43f594cf4d20 None admin] PATH_INFO: `/v3/projects/1dde52f3fffc40b1b097a718ce6aab13/users/9fe425c33e1e4fbeb81186f5585399d8/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.418012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0580a-a024-4705-a47c-43f594cf4d20 None admin] RBAC: Authorizing `identity:create_grant(project_id=1dde52f3fffc40b1b097a718ce6aab13, user_id=9fe425c33e1e4fbeb81186f5585399d8, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.420264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0580a-a024-4705-a47c-43f594cf4d20 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.440557 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 70/132] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:36 2022] PUT /identity/v3/projects/1dde52f3fffc40b1b097a718ce6aab13/users/9fe425c33e1e4fbeb81186f5585399d8/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:36.441588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0580a-a024-4705-a47c-43f594cf4d20 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.483199 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8105841-756b-4c49-bb56-a7708e6fa4bb None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.553316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ea37b-f904-4412-b12c-da7a22278cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.554379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ea37b-f904-4412-b12c-da7a22278cf5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.554830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ea37b-f904-4412-b12c-da7a22278cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.555248 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ea37b-f904-4412-b12c-da7a22278cf5 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.556700 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ea37b-f904-4412-b12c-da7a22278cf5 None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.558537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ea37b-f904-4412-b12c-da7a22278cf5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.560812 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 63/133] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:36.575138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ea37b-f904-4412-b12c-da7a22278cf5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.644088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.676815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0580a-a024-4705-a47c-43f594cf4d20 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.737206 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.760054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21d895-acf4-4421-a224-7d4e990bf3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.760936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21d895-acf4-4421-a224-7d4e990bf3f4 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.761386 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21d895-acf4-4421-a224-7d4e990bf3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.761809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ea37b-f904-4412-b12c-da7a22278cf5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.761957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21d895-acf4-4421-a224-7d4e990bf3f4 None admin] PATH_INFO: `/v3/users/9fe425c33e1e4fbeb81186f5585399d8` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.763228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21d895-acf4-4421-a224-7d4e990bf3f4 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe425c33e1e4fbeb81186f5585399d8)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.764842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21d895-acf4-4421-a224-7d4e990bf3f4 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.767003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 71/134] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/users/9fe425c33e1e4fbeb81186f5585399d8 => generated 275 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:36.773371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21d895-acf4-4421-a224-7d4e990bf3f4 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.804615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988635c2-847d-4ad2-94de-a8320207acfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.805671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988635c2-847d-4ad2-94de-a8320207acfd None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.806018 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988635c2-847d-4ad2-94de-a8320207acfd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.806302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988635c2-847d-4ad2-94de-a8320207acfd None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41/users/9fe425c33e1e4fbeb81186f5585399d8/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.808853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988635c2-847d-4ad2-94de-a8320207acfd None admin] RBAC: Authorizing `identity:create_grant(project_id=32e961e117e24576acf2592d4ee51e41, user_id=9fe425c33e1e4fbeb81186f5585399d8, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.810734 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988635c2-847d-4ad2-94de-a8320207acfd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.829660 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 64/135] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:36 2022] PUT /identity/v3/projects/32e961e117e24576acf2592d4ee51e41/users/9fe425c33e1e4fbeb81186f5585399d8/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:36.859338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1846ab-6329-41be-b0ba-aacd3105e3b4 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.859633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1846ab-6329-41be-b0ba-aacd3105e3b4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.860167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1846ab-6329-41be-b0ba-aacd3105e3b4 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.861649 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 65/136] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:36.869858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45f591-9e86-4da1-a433-fed44d3d2081 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.870148 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45f591-9e86-4da1-a433-fed44d3d2081 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.870423 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45f591-9e86-4da1-a433-fed44d3d2081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.913761 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1d45f591-9e86-4da1-a433-fed44d3d2081 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:36.913761 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1d45f591-9e86-4da1-a433-fed44d3d2081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:37.016190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21d895-acf4-4421-a224-7d4e990bf3f4 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.032855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 66/137] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:37.064032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5d341-b6c2-4e2b-980c-a5e1cc415e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.064673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5d341-b6c2-4e2b-980c-a5e1cc415e01 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.064957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5d341-b6c2-4e2b-980c-a5e1cc415e01 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.065233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5d341-b6c2-4e2b-980c-a5e1cc415e01 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.066393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5d341-b6c2-4e2b-980c-a5e1cc415e01 None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.067857 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5d341-b6c2-4e2b-980c-a5e1cc415e01 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.069679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 72/138] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:37.075669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45f591-9e86-4da1-a433-fed44d3d2081 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.098639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:37.198861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45f591-9e86-4da1-a433-fed44d3d2081 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.237499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04098b7-3509-4caf-b5c3-c7012911abb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.238209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04098b7-3509-4caf-b5c3-c7012911abb0 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.238487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04098b7-3509-4caf-b5c3-c7012911abb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.238762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04098b7-3509-4caf-b5c3-c7012911abb0 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41/users/9fe425c33e1e4fbeb81186f5585399d8/roles/7d0ac07c6114438da47dcfefeb1fe05b` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.240928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04098b7-3509-4caf-b5c3-c7012911abb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=32e961e117e24576acf2592d4ee51e41, user_id=9fe425c33e1e4fbeb81186f5585399d8, role_id=7d0ac07c6114438da47dcfefeb1fe05b)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.242428 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04098b7-3509-4caf-b5c3-c7012911abb0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.257958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 67/139] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:37 2022] PUT /identity/v3/projects/32e961e117e24576acf2592d4ee51e41/users/9fe425c33e1e4fbeb81186f5585399d8/roles/7d0ac07c6114438da47dcfefeb1fe05b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:37.483350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5d341-b6c2-4e2b-980c-a5e1cc415e01 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.563020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:37.610386 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5d341-b6c2-4e2b-980c-a5e1cc415e01 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.654522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ba20b-2ecd-407d-b72e-1ac2a7d1a7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.655240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ba20b-2ecd-407d-b72e-1ac2a7d1a7de None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.656064 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ba20b-2ecd-407d-b72e-1ac2a7d1a7de None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.656344 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ba20b-2ecd-407d-b72e-1ac2a7d1a7de None admin] PATH_INFO: `/v3/domains/default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.657559 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ba20b-2ecd-407d-b72e-1ac2a7d1a7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.659045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ba20b-2ecd-407d-b72e-1ac2a7d1a7de None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.660947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 73/140] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:37.666572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04098b7-3509-4caf-b5c3-c7012911abb0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.791397 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:37.824444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04098b7-3509-4caf-b5c3-c7012911abb0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.870451 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be1958-eecf-4d0b-ba36-7b0b71739532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.871079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be1958-eecf-4d0b-ba36-7b0b71739532 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.871362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be1958-eecf-4d0b-ba36-7b0b71739532 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.871641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be1958-eecf-4d0b-ba36-7b0b71739532 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.872312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be1958-eecf-4d0b-ba36-7b0b71739532 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.873773 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be1958-eecf-4d0b-ba36-7b0b71739532 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.904293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 68/141] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:37 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:38.784200 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34e04a-b8a1-4e95-aee0-ab853bbc4b81 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.784803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34e04a-b8a1-4e95-aee0-ab853bbc4b81 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.784803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34e04a-b8a1-4e95-aee0-ab853bbc4b81 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.786179 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 74/142] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:38.793686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96237b64-7605-4b8c-9931-207dec8be53a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.794576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96237b64-7605-4b8c-9931-207dec8be53a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.795059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96237b64-7605-4b8c-9931-207dec8be53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.816001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-96237b64-7605-4b8c-9931-207dec8be53a None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:38.817507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-96237b64-7605-4b8c-9931-207dec8be53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:38.899436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 69/143] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:39.385278 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c4540-661f-4c4d-9d7b-e165684f21bf None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.385988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c4540-661f-4c4d-9d7b-e165684f21bf None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.386500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c4540-661f-4c4d-9d7b-e165684f21bf None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.388071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 75/144] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:39.393631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d0d1d-7126-49ea-9bd9-e22e167bb00d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.393959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d0d1d-7126-49ea-9bd9-e22e167bb00d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.394243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d0d1d-7126-49ea-9bd9-e22e167bb00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.422549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ea8d0d1d-7126-49ea-9bd9-e22e167bb00d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:39.423762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ea8d0d1d-7126-49ea-9bd9-e22e167bb00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:39.489062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:39.493629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c4540-661f-4c4d-9d7b-e165684f21bf None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.511154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 70/145] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:39.623331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c4540-661f-4c4d-9d7b-e165684f21bf None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.658368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:39.664085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c47a2c-a5c0-4eda-96fd-8abacb5516f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.664744 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c47a2c-a5c0-4eda-96fd-8abacb5516f1 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.665250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c47a2c-a5c0-4eda-96fd-8abacb5516f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.665722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c47a2c-a5c0-4eda-96fd-8abacb5516f1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.685401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c47a2c-a5c0-4eda-96fd-8abacb5516f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-b2c47a2c-a5c0-4eda-96fd-8abacb5516f1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:39.690531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:39.691433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:39.692233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:39.692233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:39.692233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:39.692233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:39.692233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:39.692233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:39.692233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:39.692233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:39.692233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:39.692233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:39.692233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:38:39.692233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:38:39.692233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 76/146] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:39.699148 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d0d1d-7126-49ea-9bd9-e22e167bb00d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.823512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d0d1d-7126-49ea-9bd9-e22e167bb00d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.864037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71719e7c-7efd-477d-99f4-0da6ea791a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.864661 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71719e7c-7efd-477d-99f4-0da6ea791a0f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.864994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71719e7c-7efd-477d-99f4-0da6ea791a0f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.865274 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71719e7c-7efd-477d-99f4-0da6ea791a0f None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.865973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71719e7c-7efd-477d-99f4-0da6ea791a0f None admin] RBAC: Adding query filter params (name=reader) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:39.866276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71719e7c-7efd-477d-99f4-0da6ea791a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.867676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71719e7c-7efd-477d-99f4-0da6ea791a0f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.882187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 71/147] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:39.887821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c47a2c-a5c0-4eda-96fd-8abacb5516f1 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.011607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c47a2c-a5c0-4eda-96fd-8abacb5516f1 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.014639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e3c50-4858-49a6-90d0-6ba3abb6bd7c None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.014958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e3c50-4858-49a6-90d0-6ba3abb6bd7c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.015232 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e3c50-4858-49a6-90d0-6ba3abb6bd7c None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.016573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 72/148] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:40.021684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b799d-8f87-4dba-b662-09be942b897b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.021989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b799d-8f87-4dba-b662-09be942b897b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.022267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b799d-8f87-4dba-b662-09be942b897b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.050211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0d8b799d-8f87-4dba-b662-09be942b897b None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:40.051409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0d8b799d-8f87-4dba-b662-09be942b897b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:40.066475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524398ce-92d7-41ec-99b1-efe2963844ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.067131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524398ce-92d7-41ec-99b1-efe2963844ab None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.067468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524398ce-92d7-41ec-99b1-efe2963844ab None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.067745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524398ce-92d7-41ec-99b1-efe2963844ab None admin] PATH_INFO: `/v3/users/e58d91e284ee49b7b6cf95625c3c9dc2` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.073063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524398ce-92d7-41ec-99b1-efe2963844ab None admin] RBAC: Authorizing `identity:get_user(user_id=e58d91e284ee49b7b6cf95625c3c9dc2)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.074920 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524398ce-92d7-41ec-99b1-efe2963844ab None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.076774 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 77/149] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/users/e58d91e284ee49b7b6cf95625c3c9dc2 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:40.083659 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524398ce-92d7-41ec-99b1-efe2963844ab None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.166197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.196476 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 73/150] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:40.197355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b799d-8f87-4dba-b662-09be942b897b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.211868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.290822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524398ce-92d7-41ec-99b1-efe2963844ab None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.354013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10749aa-cf8a-495a-9266-fadfdd4ec799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.354646 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10749aa-cf8a-495a-9266-fadfdd4ec799 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.354925 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10749aa-cf8a-495a-9266-fadfdd4ec799 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.355192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10749aa-cf8a-495a-9266-fadfdd4ec799 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.357075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10749aa-cf8a-495a-9266-fadfdd4ec799 None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.359081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10749aa-cf8a-495a-9266-fadfdd4ec799 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.362068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 78/151] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:40.368506 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10749aa-cf8a-495a-9266-fadfdd4ec799 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.391755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b799d-8f87-4dba-b662-09be942b897b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.456910 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d48d2e-8b3a-48c4-a188-f9d7d8bde332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.457471 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48d2e-8b3a-48c4-a188-f9d7d8bde332 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.457765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48d2e-8b3a-48c4-a188-f9d7d8bde332 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.458035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d48d2e-8b3a-48c4-a188-f9d7d8bde332 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.477546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d48d2e-8b3a-48c4-a188-f9d7d8bde332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-46d48d2e-8b3a-48c4-a188-f9d7d8bde332 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:40.482444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:40.483317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:40.484705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:40.484705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:40.484705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:40.484705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:40.484705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:40.484705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:40.484705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:40.484705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:40.484705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:40.484705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:40.484705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:40.484705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:40.484705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 74/152] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:40.498359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d48d2e-8b3a-48c4-a188-f9d7d8bde332 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.568680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.578202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10749aa-cf8a-495a-9266-fadfdd4ec799 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.620687 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d48d2e-8b3a-48c4-a188-f9d7d8bde332 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.642776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.664647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a0fc3-623c-4534-90f5-e8284b45b3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.665574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a0fc3-623c-4534-90f5-e8284b45b3d6 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.666616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a0fc3-623c-4534-90f5-e8284b45b3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.667031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a0fc3-623c-4534-90f5-e8284b45b3d6 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.667856 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a0fc3-623c-4534-90f5-e8284b45b3d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:40.668362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a0fc3-623c-4534-90f5-e8284b45b3d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.669909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a0fc3-623c-4534-90f5-e8284b45b3d6 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.678316 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331cf01-6151-4583-bfd6-308b958f9028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331cf01-6151-4583-bfd6-308b958f9028 None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.679287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331cf01-6151-4583-bfd6-308b958f9028 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.679562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331cf01-6151-4583-bfd6-308b958f9028 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41/users/e58d91e284ee49b7b6cf95625c3c9dc2/roles/eeefde0ca97549ed9cd98300215a29ad` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.688663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 75/153] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:40.688827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331cf01-6151-4583-bfd6-308b958f9028 None admin] RBAC: Authorizing `identity:create_grant(project_id=32e961e117e24576acf2592d4ee51e41, user_id=e58d91e284ee49b7b6cf95625c3c9dc2, role_id=eeefde0ca97549ed9cd98300215a29ad)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.690243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2587d92-335d-4759-9f7f-34fa86d8a513 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.690371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331cf01-6151-4583-bfd6-308b958f9028 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.690783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2587d92-335d-4759-9f7f-34fa86d8a513 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.691213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2587d92-335d-4759-9f7f-34fa86d8a513 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.692671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 76/154] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:40.698344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2587d92-335d-4759-9f7f-34fa86d8a513 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.717059 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 79/155] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:40 2022] PUT /identity/v3/projects/32e961e117e24576acf2592d4ee51e41/users/e58d91e284ee49b7b6cf95625c3c9dc2/roles/eeefde0ca97549ed9cd98300215a29ad => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:40.717773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f844d8f-9361-47ab-82b2-6a60558d2600 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.718068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f844d8f-9361-47ab-82b2-6a60558d2600 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.718404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f844d8f-9361-47ab-82b2-6a60558d2600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.754922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-0f844d8f-9361-47ab-82b2-6a60558d2600 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:40.756278 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0f844d8f-9361-47ab-82b2-6a60558d2600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:40.894641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2587d92-335d-4759-9f7f-34fa86d8a513 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.907608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 80/156] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:40.909096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f844d8f-9361-47ab-82b2-6a60558d2600 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.940493 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.958434 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638ef82-25f1-4bc2-be84-5bfd4d87f10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.959094 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638ef82-25f1-4bc2-be84-5bfd4d87f10d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.959387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638ef82-25f1-4bc2-be84-5bfd4d87f10d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.959642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638ef82-25f1-4bc2-be84-5bfd4d87f10d None admin] PATH_INFO: `/v3/users/3709ff21e04749f7aae9073ca845e875` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.960802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638ef82-25f1-4bc2-be84-5bfd4d87f10d None admin] RBAC: Authorizing `identity:get_user(user_id=3709ff21e04749f7aae9073ca845e875)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.970587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638ef82-25f1-4bc2-be84-5bfd4d87f10d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.972514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 77/157] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/users/3709ff21e04749f7aae9073ca845e875 => generated 247 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:40.978186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638ef82-25f1-4bc2-be84-5bfd4d87f10d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.028434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f844d8f-9361-47ab-82b2-6a60558d2600 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.047523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.071334 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55600c76-a44b-4760-8c53-fd48c4d5c740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.072064 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55600c76-a44b-4760-8c53-fd48c4d5c740 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.072343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55600c76-a44b-4760-8c53-fd48c4d5c740 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.072618 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55600c76-a44b-4760-8c53-fd48c4d5c740 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.084588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55600c76-a44b-4760-8c53-fd48c4d5c740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-55600c76-a44b-4760-8c53-fd48c4d5c740 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:41.093939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:41.094961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:41.095779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:41.095779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:41.095779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:41.095779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:41.095779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:41.095779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:41.095779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:41.095779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:41.095779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:41.095779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:41.095779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:41.095779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:38:41.095779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 81/158] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:41.104138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55600c76-a44b-4760-8c53-fd48c4d5c740 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.153336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638ef82-25f1-4bc2-be84-5bfd4d87f10d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.220060 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50210b70-4f05-4b92-9a9a-cb720ef98021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.220766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50210b70-4f05-4b92-9a9a-cb720ef98021 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.220985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50210b70-4f05-4b92-9a9a-cb720ef98021 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.221251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50210b70-4f05-4b92-9a9a-cb720ef98021 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.259033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55600c76-a44b-4760-8c53-fd48c4d5c740 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.297717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.301669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479eac18-48f0-41f6-8982-069bff2ad871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.302333 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eac18-48f0-41f6-8982-069bff2ad871 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.302889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eac18-48f0-41f6-8982-069bff2ad871 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.302889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479eac18-48f0-41f6-8982-069bff2ad871 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.303570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479eac18-48f0-41f6-8982-069bff2ad871 None admin] RBAC: Adding query filter params (name=member) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:41.303954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479eac18-48f0-41f6-8982-069bff2ad871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.305417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479eac18-48f0-41f6-8982-069bff2ad871 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.314467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 82/159] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:41.320112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479eac18-48f0-41f6-8982-069bff2ad871 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.326161 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50210b70-4f05-4b92-9a9a-cb720ef98021 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.328226 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50210b70-4f05-4b92-9a9a-cb720ef98021 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.365767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.433775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479eac18-48f0-41f6-8982-069bff2ad871 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.443419 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 78/160] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:38:41.449344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50210b70-4f05-4b92-9a9a-cb720ef98021 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.470677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bce141-bf65-4e10-b2a3-e3e82379fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.471378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bce141-bf65-4e10-b2a3-e3e82379fa6d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.471601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bce141-bf65-4e10-b2a3-e3e82379fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.471875 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bce141-bf65-4e10-b2a3-e3e82379fa6d None admin] PATH_INFO: `/v3/users/9fe425c33e1e4fbeb81186f5585399d8` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.473079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bce141-bf65-4e10-b2a3-e3e82379fa6d None admin] RBAC: Authorizing `identity:get_user(user_id=9fe425c33e1e4fbeb81186f5585399d8)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.474561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bce141-bf65-4e10-b2a3-e3e82379fa6d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.476145 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 83/161] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/users/9fe425c33e1e4fbeb81186f5585399d8 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:41.481471 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bce141-bf65-4e10-b2a3-e3e82379fa6d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.542690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.552239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50210b70-4f05-4b92-9a9a-cb720ef98021 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.588209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbd91a-6aa5-4b21-828d-ca12b1f1c843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.588777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bce141-bf65-4e10-b2a3-e3e82379fa6d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.588909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbd91a-6aa5-4b21-828d-ca12b1f1c843 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.589135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbd91a-6aa5-4b21-828d-ca12b1f1c843 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.589409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbd91a-6aa5-4b21-828d-ca12b1f1c843 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.590567 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbd91a-6aa5-4b21-828d-ca12b1f1c843 None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.592024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbd91a-6aa5-4b21-828d-ca12b1f1c843 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.595111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 79/162] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:41.600123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.600386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbd91a-6aa5-4b21-828d-ca12b1f1c843 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.628995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffb7c2-648b-49ff-bc8a-3baa9ab869d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.629616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffb7c2-648b-49ff-bc8a-3baa9ab869d5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.629927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffb7c2-648b-49ff-bc8a-3baa9ab869d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.630199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ffb7c2-648b-49ff-bc8a-3baa9ab869d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.702686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbd91a-6aa5-4b21-828d-ca12b1f1c843 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.706590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ffb7c2-648b-49ff-bc8a-3baa9ab869d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.708272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ffb7c2-648b-49ff-bc8a-3baa9ab869d5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.737886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602f12e-3707-475a-afee-3874beb75707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.738420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602f12e-3707-475a-afee-3874beb75707 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.738698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602f12e-3707-475a-afee-3874beb75707 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.738969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602f12e-3707-475a-afee-3874beb75707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.739703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602f12e-3707-475a-afee-3874beb75707 None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=32e961e117e24576acf2592d4ee51e41, user.id=3709ff21e04749f7aae9073ca845e875) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:41.740008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602f12e-3707-475a-afee-3874beb75707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.741459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602f12e-3707-475a-afee-3874beb75707 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.747952 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 80/163] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/role_assignments?user.id=3709ff21e04749f7aae9073ca845e875&scope.project.id=32e961e117e24576acf2592d4ee51e41&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:41.748773 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602f12e-3707-475a-afee-3874beb75707 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.771353 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.908629 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602f12e-3707-475a-afee-3874beb75707 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.912249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 84/164] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:38:41.922792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffb7c2-648b-49ff-bc8a-3baa9ab869d5 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.940331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.971272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d424978-44ca-4c0d-8301-f790380bba86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.971925 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d424978-44ca-4c0d-8301-f790380bba86 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.972209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d424978-44ca-4c0d-8301-f790380bba86 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.972479 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d424978-44ca-4c0d-8301-f790380bba86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.984356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d424978-44ca-4c0d-8301-f790380bba86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1d424978-44ca-4c0d-8301-f790380bba86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:41.989827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:41.990698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:41.991736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:41.991736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:41.991736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:41.991736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:41.991736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:41.991736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:41.991736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:41.991736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:41.991736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:41.991736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:41.991736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:41.991736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:41.995332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 81/165] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:42.000907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d424978-44ca-4c0d-8301-f790380bba86 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.077790 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.110117 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d424978-44ca-4c0d-8301-f790380bba86 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.147008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acfb7d-b61e-4c8b-9d6f-f43dbe43a072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.147630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acfb7d-b61e-4c8b-9d6f-f43dbe43a072 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.147906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acfb7d-b61e-4c8b-9d6f-f43dbe43a072 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.148177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acfb7d-b61e-4c8b-9d6f-f43dbe43a072 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.148850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acfb7d-b61e-4c8b-9d6f-f43dbe43a072 None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:42.149158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acfb7d-b61e-4c8b-9d6f-f43dbe43a072 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.150602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acfb7d-b61e-4c8b-9d6f-f43dbe43a072 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.160262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 82/166] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:42.161356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458cb9a-1e3e-48d9-a58b-2779dcdf683b None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.161645 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458cb9a-1e3e-48d9-a58b-2779dcdf683b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.161943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458cb9a-1e3e-48d9-a58b-2779dcdf683b None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.163277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 83/167] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:42.166630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458cb9a-1e3e-48d9-a58b-2779dcdf683b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.167439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ffb7c2-648b-49ff-bc8a-3baa9ab869d5 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.268018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1b069-e81f-4ebc-837a-d1c048608ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.269035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1b069-e81f-4ebc-837a-d1c048608ad1 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.269556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1b069-e81f-4ebc-837a-d1c048608ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.270073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1b069-e81f-4ebc-837a-d1c048608ad1 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.271621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1b069-e81f-4ebc-837a-d1c048608ad1 None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.273543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1b069-e81f-4ebc-837a-d1c048608ad1 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.275566 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 85/168] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:42.278164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53c6c-01ba-4b6e-b551-6e769e903c51 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.278681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53c6c-01ba-4b6e-b551-6e769e903c51 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.279185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac53c6c-01ba-4b6e-b551-6e769e903c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.283769 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458cb9a-1e3e-48d9-a58b-2779dcdf683b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.315815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-eac53c6c-01ba-4b6e-b551-6e769e903c51 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:42.317354 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-eac53c6c-01ba-4b6e-b551-6e769e903c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:42.325765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.325765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77489722-80de-4e63-ada2-c81916db2484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.325765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77489722-80de-4e63-ada2-c81916db2484 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.325765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77489722-80de-4e63-ada2-c81916db2484 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.326382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77489722-80de-4e63-ada2-c81916db2484 None admin] PATH_INFO: `/v3/users/9fe425c33e1e4fbeb81186f5585399d8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.326382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77489722-80de-4e63-ada2-c81916db2484 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe425c33e1e4fbeb81186f5585399d8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.327178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77489722-80de-4e63-ada2-c81916db2484 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.328770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 84/169] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/users/9fe425c33e1e4fbeb81186f5585399d8 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:42.334270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77489722-80de-4e63-ada2-c81916db2484 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.446301 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77489722-80de-4e63-ada2-c81916db2484 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.448091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 86/170] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:42.449045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac53c6c-01ba-4b6e-b551-6e769e903c51 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.462581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.503657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572eb030-a5c5-4db9-aa6e-601c50281150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.504521 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eb030-a5c5-4db9-aa6e-601c50281150 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.508912 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eb030-a5c5-4db9-aa6e-601c50281150 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.509356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eb030-a5c5-4db9-aa6e-601c50281150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.510335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572eb030-a5c5-4db9-aa6e-601c50281150 None admin] RBAC: Adding query filter params (role.id=8abe8932c5974e7db6fb974c588a9d70, scope.project.id=32e961e117e24576acf2592d4ee51e41, user.id=9fe425c33e1e4fbeb81186f5585399d8) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:42.510810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572eb030-a5c5-4db9-aa6e-601c50281150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.512401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572eb030-a5c5-4db9-aa6e-601c50281150 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.524498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 85/171] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/role_assignments?user.id=9fe425c33e1e4fbeb81186f5585399d8&scope.project.id=32e961e117e24576acf2592d4ee51e41&role.id=8abe8932c5974e7db6fb974c588a9d70 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:42.631786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac53c6c-01ba-4b6e-b551-6e769e903c51 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.679861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebec222-84bd-4a42-a4be-ad83bae43a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.680485 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebec222-84bd-4a42-a4be-ad83bae43a15 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.680768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebec222-84bd-4a42-a4be-ad83bae43a15 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.681039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebec222-84bd-4a42-a4be-ad83bae43a15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.765778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebec222-84bd-4a42-a4be-ad83bae43a15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.767543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebec222-84bd-4a42-a4be-ad83bae43a15 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.779423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.896389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 87/172] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:42.912957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572eb030-a5c5-4db9-aa6e-601c50281150 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.007195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebec222-84bd-4a42-a4be-ad83bae43a15 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.010664 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.072682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572eb030-a5c5-4db9-aa6e-601c50281150 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.134982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f79143-ee0f-44af-a1b0-4ee65be3de3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.135994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f79143-ee0f-44af-a1b0-4ee65be3de3b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.136490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f79143-ee0f-44af-a1b0-4ee65be3de3b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.136969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f79143-ee0f-44af-a1b0-4ee65be3de3b None admin] PATH_INFO: `/v3/projects/1dde52f3fffc40b1b097a718ce6aab13` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.138418 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f79143-ee0f-44af-a1b0-4ee65be3de3b None admin] RBAC: Authorizing `identity:get_project(project_id=1dde52f3fffc40b1b097a718ce6aab13)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.140150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f79143-ee0f-44af-a1b0-4ee65be3de3b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.142322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 86/173] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/projects/1dde52f3fffc40b1b097a718ce6aab13 => generated 312 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:43.143851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01279ea5-deb8-4f51-97a6-a44550117859 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.144433 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01279ea5-deb8-4f51-97a6-a44550117859 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.145009 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01279ea5-deb8-4f51-97a6-a44550117859 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.146578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 87/174] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:43.147936 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01279ea5-deb8-4f51-97a6-a44550117859 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.197936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebec222-84bd-4a42-a4be-ad83bae43a15 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.216578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.271988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3f7fb-f301-4e5e-a0a7-6dad77fe9151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.275118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3f7fb-f301-4e5e-a0a7-6dad77fe9151 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.275717 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3f7fb-f301-4e5e-a0a7-6dad77fe9151 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.276879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3f7fb-f301-4e5e-a0a7-6dad77fe9151 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.300669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c3f7fb-f301-4e5e-a0a7-6dad77fe9151 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-29c3f7fb-f301-4e5e-a0a7-6dad77fe9151 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:43.309487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:43.310456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:43.311326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:43.311326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:43.311326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:43.311326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:43.311326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:43.311326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:43.311326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:43.311326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:43.311326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:43.311326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:43.311326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:43.311326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:38:43.318284 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 88/175] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:43.321625 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06a44a-56c1-4127-ac19-cf42ebb46654 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.322360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06a44a-56c1-4127-ac19-cf42ebb46654 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.323821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06a44a-56c1-4127-ac19-cf42ebb46654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.363001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01279ea5-deb8-4f51-97a6-a44550117859 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.366273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-ee06a44a-56c1-4127-ac19-cf42ebb46654 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:43.370094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ee06a44a-56c1-4127-ac19-cf42ebb46654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:43.414623 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.422081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fc096-08ab-4347-a480-a2af885ad655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.423097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fc096-08ab-4347-a480-a2af885ad655 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.423574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fc096-08ab-4347-a480-a2af885ad655 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.424066 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531fc096-08ab-4347-a480-a2af885ad655 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.425044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531fc096-08ab-4347-a480-a2af885ad655 None admin] RBAC: Adding query filter params (role.id=8abe8932c5974e7db6fb974c588a9d70, scope.project.id=1dde52f3fffc40b1b097a718ce6aab13, user.id=9fe425c33e1e4fbeb81186f5585399d8) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:43.425646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531fc096-08ab-4347-a480-a2af885ad655 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.428386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531fc096-08ab-4347-a480-a2af885ad655 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.438496 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 88/176] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/role_assignments?user.id=9fe425c33e1e4fbeb81186f5585399d8&scope.project.id=1dde52f3fffc40b1b097a718ce6aab13&role.id=8abe8932c5974e7db6fb974c588a9d70 => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:43.440584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fc096-08ab-4347-a480-a2af885ad655 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.524919 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 89/177] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:43.599981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531fc096-08ab-4347-a480-a2af885ad655 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.602524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c61cf-7b13-4d84-82e5-9e37741234f2 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.602825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c61cf-7b13-4d84-82e5-9e37741234f2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.603105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c61cf-7b13-4d84-82e5-9e37741234f2 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.610734 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 90/178] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:43.620420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb87ae-9327-4d1a-9ffe-a21204116db9 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.620766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb87ae-9327-4d1a-9ffe-a21204116db9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.620977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb87ae-9327-4d1a-9ffe-a21204116db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.653483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e9916-3e5d-40fb-9a9d-b9d9cc4811e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.654557 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e9916-3e5d-40fb-9a9d-b9d9cc4811e6 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.655636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e9916-3e5d-40fb-9a9d-b9d9cc4811e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.656486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e9916-3e5d-40fb-9a9d-b9d9cc4811e6 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.657443 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e9916-3e5d-40fb-9a9d-b9d9cc4811e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:43.657984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e9916-3e5d-40fb-9a9d-b9d9cc4811e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.659907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e9916-3e5d-40fb-9a9d-b9d9cc4811e6 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.660374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-76bb87ae-9327-4d1a-9ffe-a21204116db9 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:43.661606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-76bb87ae-9327-4d1a-9ffe-a21204116db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:43.676241 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 89/179] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:43.688327 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e9916-3e5d-40fb-9a9d-b9d9cc4811e6 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.782292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 91/180] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:43.783347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.814783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e2e0-15e0-4281-9869-2eda1d7e2f66 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.815439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e2e0-15e0-4281-9869-2eda1d7e2f66 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.815918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0e2e0-15e0-4281-9869-2eda1d7e2f66 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.817718 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 92/181] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:43.823484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b5386-31fa-490b-9d67-9c008e86911d None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.824001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b5386-31fa-490b-9d67-9c008e86911d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.824565 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b5386-31fa-490b-9d67-9c008e86911d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.845149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e9916-3e5d-40fb-9a9d-b9d9cc4811e6 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.861201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-cb0b5386-31fa-490b-9d67-9c008e86911d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:43.862983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-cb0b5386-31fa-490b-9d67-9c008e86911d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:43.905893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.921755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d2dab-4e80-424c-827f-cfab07cd4a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.921755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d2dab-4e80-424c-827f-cfab07cd4a65 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.921755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d2dab-4e80-424c-827f-cfab07cd4a65 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.921755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d2dab-4e80-424c-827f-cfab07cd4a65 None admin] PATH_INFO: `/v3/users/9fe425c33e1e4fbeb81186f5585399d8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.922421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d2dab-4e80-424c-827f-cfab07cd4a65 None admin] RBAC: Authorizing `identity:get_user(user_id=9fe425c33e1e4fbeb81186f5585399d8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.922786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d2dab-4e80-424c-827f-cfab07cd4a65 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.924548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 90/182] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/users/9fe425c33e1e4fbeb81186f5585399d8 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:43.928881 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d2dab-4e80-424c-827f-cfab07cd4a65 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.975592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 93/183] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:43.976468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b5386-31fa-490b-9d67-9c008e86911d None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.073021 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d2dab-4e80-424c-827f-cfab07cd4a65 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.097478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b5386-31fa-490b-9d67-9c008e86911d None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.105067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.122235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744be47-f3a9-407b-bea1-9a88766fddc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.122878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744be47-f3a9-407b-bea1-9a88766fddc5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.123154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744be47-f3a9-407b-bea1-9a88766fddc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.123436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744be47-f3a9-407b-bea1-9a88766fddc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.137005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744be47-f3a9-407b-bea1-9a88766fddc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-7744be47-f3a9-407b-bea1-9a88766fddc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:44.142968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:44.143874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:44.144690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:44.144690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:44.144690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:44.144690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:44.144690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:44.144690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:44.144690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:44.144690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:44.144690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:44.144690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:44.144690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:44.144690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:44.144690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 91/184] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:44.148925 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edf9e2-bee9-4e6d-b3d2-2e14564eff84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.149780 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744be47-f3a9-407b-bea1-9a88766fddc5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.150043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edf9e2-bee9-4e6d-b3d2-2e14564eff84 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.150532 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edf9e2-bee9-4e6d-b3d2-2e14564eff84 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.151015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edf9e2-bee9-4e6d-b3d2-2e14564eff84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.190078 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.234614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edf9e2-bee9-4e6d-b3d2-2e14564eff84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.236516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edf9e2-bee9-4e6d-b3d2-2e14564eff84 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.257411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744be47-f3a9-407b-bea1-9a88766fddc5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.309027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b733c02-af51-4ad2-8add-b261a53f7113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.309642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b733c02-af51-4ad2-8add-b261a53f7113 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.310422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b733c02-af51-4ad2-8add-b261a53f7113 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.310708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b733c02-af51-4ad2-8add-b261a53f7113 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.311963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b733c02-af51-4ad2-8add-b261a53f7113 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:44.312533 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b733c02-af51-4ad2-8add-b261a53f7113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.314207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b733c02-af51-4ad2-8add-b261a53f7113 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.325790 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 92/185] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:44.326600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b733c02-af51-4ad2-8add-b261a53f7113 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.362784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 94/186] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:44.364046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edf9e2-bee9-4e6d-b3d2-2e14564eff84 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.391430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.426624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.451975 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b733c02-af51-4ad2-8add-b261a53f7113 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.472065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edf9e2-bee9-4e6d-b3d2-2e14564eff84 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.488764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5170bf-9d5c-45d6-adb2-058f6d8dbf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.489401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5170bf-9d5c-45d6-adb2-058f6d8dbf30 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.489680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5170bf-9d5c-45d6-adb2-058f6d8dbf30 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.490079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5170bf-9d5c-45d6-adb2-058f6d8dbf30 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.501052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5170bf-9d5c-45d6-adb2-058f6d8dbf30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-8d5170bf-9d5c-45d6-adb2-058f6d8dbf30 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:44.506669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:44.507643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:44.508440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:44.508440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:44.508440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:44.508440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:44.508440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:44.508440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:44.508440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:44.508440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:44.508440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:44.508440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:44.508440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:44.508440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:44.509041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6bc60-cfcf-4943-b253-cf0b8efdd3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.509041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6bc60-cfcf-4943-b253-cf0b8efdd3bf None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.509199 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 93/187] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:44.509267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6bc60-cfcf-4943-b253-cf0b8efdd3bf None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.509267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6bc60-cfcf-4943-b253-cf0b8efdd3bf None admin] PATH_INFO: `/v3/users/b6ffdb84fa4a4add84fbcc8070f44a67` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.509384 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5170bf-9d5c-45d6-adb2-058f6d8dbf30 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.528267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6bc60-cfcf-4943-b253-cf0b8efdd3bf None admin] RBAC: Authorizing `identity:get_user(user_id=b6ffdb84fa4a4add84fbcc8070f44a67)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.529770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6bc60-cfcf-4943-b253-cf0b8efdd3bf None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.532001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 95/188] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/users/b6ffdb84fa4a4add84fbcc8070f44a67 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:44.532001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6bc60-cfcf-4943-b253-cf0b8efdd3bf None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.626672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5170bf-9d5c-45d6-adb2-058f6d8dbf30 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.636203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.639247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6bc60-cfcf-4943-b253-cf0b8efdd3bf None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.653911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.666472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2af69-7940-4857-af63-1d7f4c2bb13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.669182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2af69-7940-4857-af63-1d7f4c2bb13a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.669182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2af69-7940-4857-af63-1d7f4c2bb13a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.669182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2af69-7940-4857-af63-1d7f4c2bb13a None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.672151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2af69-7940-4857-af63-1d7f4c2bb13a None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.672151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2af69-7940-4857-af63-1d7f4c2bb13a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.673587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 94/189] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:44.674169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2af69-7940-4857-af63-1d7f4c2bb13a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.690967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b133c-d717-4f16-a56c-dcb278de8d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.691603 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b133c-d717-4f16-a56c-dcb278de8d9b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.691941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b133c-d717-4f16-a56c-dcb278de8d9b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.692216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b133c-d717-4f16-a56c-dcb278de8d9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.693337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b133c-d717-4f16-a56c-dcb278de8d9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.694814 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b133c-d717-4f16-a56c-dcb278de8d9b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.697779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 96/190] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:44.697779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b133c-d717-4f16-a56c-dcb278de8d9b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.783672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2af69-7940-4857-af63-1d7f4c2bb13a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.805827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b133c-d717-4f16-a56c-dcb278de8d9b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.821549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd25bfa-214f-45a1-ad81-446e94ba074d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.822195 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd25bfa-214f-45a1-ad81-446e94ba074d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.822471 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd25bfa-214f-45a1-ad81-446e94ba074d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.822736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd25bfa-214f-45a1-ad81-446e94ba074d None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.823523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd25bfa-214f-45a1-ad81-446e94ba074d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:44.823946 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd25bfa-214f-45a1-ad81-446e94ba074d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.825320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd25bfa-214f-45a1-ad81-446e94ba074d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.834159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 95/191] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:44.834953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd25bfa-214f-45a1-ad81-446e94ba074d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.849040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bac53c-8083-42a2-847b-b0ee2b9544b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.849925 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bac53c-8083-42a2-847b-b0ee2b9544b1 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.850356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bac53c-8083-42a2-847b-b0ee2b9544b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.850764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bac53c-8083-42a2-847b-b0ee2b9544b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.880395 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.915038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.934895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bac53c-8083-42a2-847b-b0ee2b9544b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.936408 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bac53c-8083-42a2-847b-b0ee2b9544b1 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.941199 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd25bfa-214f-45a1-ad81-446e94ba074d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.973160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9801a-f876-4370-81be-a995c1f6286c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.973781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9801a-f876-4370-81be-a995c1f6286c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.974057 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9801a-f876-4370-81be-a995c1f6286c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.974322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9801a-f876-4370-81be-a995c1f6286c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.975031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9801a-f876-4370-81be-a995c1f6286c None admin] RBAC: Adding query filter params (role.id=7d0ac07c6114438da47dcfefeb1fe05b, scope.project.id=32e961e117e24576acf2592d4ee51e41, user.id=9fe425c33e1e4fbeb81186f5585399d8) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:44.975331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9801a-f876-4370-81be-a995c1f6286c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.976748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9801a-f876-4370-81be-a995c1f6286c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.982315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 96/192] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/role_assignments?user.id=9fe425c33e1e4fbeb81186f5585399d8&scope.project.id=32e961e117e24576acf2592d4ee51e41&role.id=7d0ac07c6114438da47dcfefeb1fe05b => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:44.984230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9801a-f876-4370-81be-a995c1f6286c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.048227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 97/193] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:45.053630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bac53c-8083-42a2-847b-b0ee2b9544b1 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.109598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9801a-f876-4370-81be-a995c1f6286c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.111272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:45.148225 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492f1aa-2dd6-4ff5-9052-210ac8ba8ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.148851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492f1aa-2dd6-4ff5-9052-210ac8ba8ede None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.149129 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492f1aa-2dd6-4ff5-9052-210ac8ba8ede None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.149701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492f1aa-2dd6-4ff5-9052-210ac8ba8ede None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.150101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492f1aa-2dd6-4ff5-9052-210ac8ba8ede None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.151547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1492f1aa-2dd6-4ff5-9052-210ac8ba8ede None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.155933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:45.176079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 97/194] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:44 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:45.177541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f9a6c-a4b7-4b41-a926-ed762c68eeca None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.178035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f9a6c-a4b7-4b41-a926-ed762c68eeca None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.178468 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f9a6c-a4b7-4b41-a926-ed762c68eeca None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.179913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 98/195] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:45.185008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f9a6c-a4b7-4b41-a926-ed762c68eeca None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.185221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bac53c-8083-42a2-847b-b0ee2b9544b1 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.230012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce27d5-aa0b-4b3c-8972-420b5524f7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.231153 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce27d5-aa0b-4b3c-8972-420b5524f7c2 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.231591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce27d5-aa0b-4b3c-8972-420b5524f7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.232380 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce27d5-aa0b-4b3c-8972-420b5524f7c2 None admin] PATH_INFO: `/v3/users/b6ffdb84fa4a4add84fbcc8070f44a67` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.234215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce27d5-aa0b-4b3c-8972-420b5524f7c2 None admin] RBAC: Authorizing `identity:get_user(user_id=b6ffdb84fa4a4add84fbcc8070f44a67)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.235900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce27d5-aa0b-4b3c-8972-420b5524f7c2 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.237828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 98/196] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/users/b6ffdb84fa4a4add84fbcc8070f44a67 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:45.240195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cde2a4-a14d-445c-b178-300a0133e525 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.240624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cde2a4-a14d-445c-b178-300a0133e525 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.241041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cde2a4-a14d-445c-b178-300a0133e525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.264061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-66cde2a4-a14d-445c-b178-300a0133e525 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:45.265362 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-66cde2a4-a14d-445c-b178-300a0133e525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:45.345067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f9a6c-a4b7-4b41-a926-ed762c68eeca None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.353926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 99/197] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:45.354884 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cde2a4-a14d-445c-b178-300a0133e525 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.394820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54673b-53a9-4950-9c05-5d94475a68e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.395736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54673b-53a9-4950-9c05-5d94475a68e2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.396021 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54673b-53a9-4950-9c05-5d94475a68e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.396295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54673b-53a9-4950-9c05-5d94475a68e2 None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.397500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54673b-53a9-4950-9c05-5d94475a68e2 None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.399046 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d54673b-53a9-4950-9c05-5d94475a68e2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.401097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 99/198] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:45.407239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54673b-53a9-4950-9c05-5d94475a68e2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.442802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:45.461929 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:45.486241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cde2a4-a14d-445c-b178-300a0133e525 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.534670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54673b-53a9-4950-9c05-5d94475a68e2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.538374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721d6e4-561c-4627-a535-4ad4738a3c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.539258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721d6e4-561c-4627-a535-4ad4738a3c4e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.539729 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721d6e4-561c-4627-a535-4ad4738a3c4e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.540206 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721d6e4-561c-4627-a535-4ad4738a3c4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.580203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16ff2c-a0bc-41b2-a7a8-51d78d375c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.580850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16ff2c-a0bc-41b2-a7a8-51d78d375c73 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.581131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16ff2c-a0bc-41b2-a7a8-51d78d375c73 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.581402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16ff2c-a0bc-41b2-a7a8-51d78d375c73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.582164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16ff2c-a0bc-41b2-a7a8-51d78d375c73 None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd, user.id=b6ffdb84fa4a4add84fbcc8070f44a67) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:45.582473 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16ff2c-a0bc-41b2-a7a8-51d78d375c73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.583958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16ff2c-a0bc-41b2-a7a8-51d78d375c73 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.590672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 100/199] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/role_assignments?user.id=b6ffdb84fa4a4add84fbcc8070f44a67&scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:45.630733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721d6e4-561c-4627-a535-4ad4738a3c4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.632233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721d6e4-561c-4627-a535-4ad4738a3c4e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.646445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16ff2c-a0bc-41b2-a7a8-51d78d375c73 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.734601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:45.759562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 100/200] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:45.765149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721d6e4-561c-4627-a535-4ad4738a3c4e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.787456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16ff2c-a0bc-41b2-a7a8-51d78d375c73 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.798688 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:45.856869 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2063c5-86b5-4fea-aafc-ab08f34557e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.857523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2063c5-86b5-4fea-aafc-ab08f34557e7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.857826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2063c5-86b5-4fea-aafc-ab08f34557e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.858678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2063c5-86b5-4fea-aafc-ab08f34557e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.871119 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2063c5-86b5-4fea-aafc-ab08f34557e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-0e2063c5-86b5-4fea-aafc-ab08f34557e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:45.878394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:45.879315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:45.880145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:45.880145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:45.880145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:45.880145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:45.880145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:45.880145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:45.880145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:45.880145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:45.880145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:45.880145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:45.880145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:38:45.880145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:45.880145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 101/201] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:45.890179 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2063c5-86b5-4fea-aafc-ab08f34557e7 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.915733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721d6e4-561c-4627-a535-4ad4738a3c4e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.954943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf466189-db76-4121-b1a7-5b318524e30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.955935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf466189-db76-4121-b1a7-5b318524e30d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.956432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf466189-db76-4121-b1a7-5b318524e30d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.956965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf466189-db76-4121-b1a7-5b318524e30d None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.958478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf466189-db76-4121-b1a7-5b318524e30d None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.960125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf466189-db76-4121-b1a7-5b318524e30d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.962014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 101/202] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:45.970666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf466189-db76-4121-b1a7-5b318524e30d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.041708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2063c5-86b5-4fea-aafc-ab08f34557e7 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.080813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:46.089159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2f105-eb9c-451a-b8df-26bab8e21e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.089830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2f105-eb9c-451a-b8df-26bab8e21e8b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.090095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2f105-eb9c-451a-b8df-26bab8e21e8b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.090367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2f105-eb9c-451a-b8df-26bab8e21e8b None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.091047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2f105-eb9c-451a-b8df-26bab8e21e8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:46.091350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2f105-eb9c-451a-b8df-26bab8e21e8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:46.094576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2f105-eb9c-451a-b8df-26bab8e21e8b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:46.113026 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 102/203] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:46.118567 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2f105-eb9c-451a-b8df-26bab8e21e8b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.145940 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:46.163209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf466189-db76-4121-b1a7-5b318524e30d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.229640 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77cc3f-f966-4e98-be25-940c79cf1fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.238205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77cc3f-f966-4e98-be25-940c79cf1fe6 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.238492 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77cc3f-f966-4e98-be25-940c79cf1fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.238768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec77cc3f-f966-4e98-be25-940c79cf1fe6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.239525 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77cc3f-f966-4e98-be25-940c79cf1fe6 None admin] RBAC: Adding query filter params (role.id=7d0ac07c6114438da47dcfefeb1fe05b, scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd, user.id=b6ffdb84fa4a4add84fbcc8070f44a67) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:46.239839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77cc3f-f966-4e98-be25-940c79cf1fe6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:46.241301 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec77cc3f-f966-4e98-be25-940c79cf1fe6 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:46.254459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 102/204] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/role_assignments?user.id=b6ffdb84fa4a4add84fbcc8070f44a67&scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd&role.id=7d0ac07c6114438da47dcfefeb1fe05b => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:46.294362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2f105-eb9c-451a-b8df-26bab8e21e8b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.333666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0fd73-dcd7-49bb-a96d-6c83bf518888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.334342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0fd73-dcd7-49bb-a96d-6c83bf518888 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.334625 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0fd73-dcd7-49bb-a96d-6c83bf518888 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.334901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc0fd73-dcd7-49bb-a96d-6c83bf518888 None admin] PATH_INFO: `/v3/users/e58d91e284ee49b7b6cf95625c3c9dc2` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.336071 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0fd73-dcd7-49bb-a96d-6c83bf518888 None admin] RBAC: Authorizing `identity:get_user(user_id=e58d91e284ee49b7b6cf95625c3c9dc2)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:46.337531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc0fd73-dcd7-49bb-a96d-6c83bf518888 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:46.339367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 103/205] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:46 2022] GET /identity/v3/users/e58d91e284ee49b7b6cf95625c3c9dc2 => generated 289 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:46.346280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77cc3f-f966-4e98-be25-940c79cf1fe6 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.468919 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec77cc3f-f966-4e98-be25-940c79cf1fe6 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.484104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:46.507602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2850f-2db2-4ed2-83e6-cb9f98a55d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.508222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2850f-2db2-4ed2-83e6-cb9f98a55d00 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.508501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2850f-2db2-4ed2-83e6-cb9f98a55d00 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.508773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2850f-2db2-4ed2-83e6-cb9f98a55d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.587362 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2850f-2db2-4ed2-83e6-cb9f98a55d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:46.588774 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2850f-2db2-4ed2-83e6-cb9f98a55d00 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:46.694514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 103/206] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:46.700474 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0fd73-dcd7-49bb-a96d-6c83bf518888 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.795118 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:46.829521 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc0fd73-dcd7-49bb-a96d-6c83bf518888 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.865084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df141e3f-c6e2-4e4e-b17f-0be8442e71fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.865721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df141e3f-c6e2-4e4e-b17f-0be8442e71fb None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.866024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df141e3f-c6e2-4e4e-b17f-0be8442e71fb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.866295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df141e3f-c6e2-4e4e-b17f-0be8442e71fb None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.867421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df141e3f-c6e2-4e4e-b17f-0be8442e71fb None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:46.868911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df141e3f-c6e2-4e4e-b17f-0be8442e71fb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:46.870610 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 104/207] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:46 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:46.877161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2850f-2db2-4ed2-83e6-cb9f98a55d00 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.894387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:47.007965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2850f-2db2-4ed2-83e6-cb9f98a55d00 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.043662 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf7680-a5ec-4427-b35b-c427d06eaf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.044281 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf7680-a5ec-4427-b35b-c427d06eaf0d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.044560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf7680-a5ec-4427-b35b-c427d06eaf0d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.044829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf7680-a5ec-4427-b35b-c427d06eaf0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.046110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf7680-a5ec-4427-b35b-c427d06eaf0d None admin] RBAC: Adding query filter params (role.id=eeefde0ca97549ed9cd98300215a29ad, scope.project.id=32e961e117e24576acf2592d4ee51e41, user.id=e58d91e284ee49b7b6cf95625c3c9dc2) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:47.046421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf7680-a5ec-4427-b35b-c427d06eaf0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.047861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcf7680-a5ec-4427-b35b-c427d06eaf0d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.054167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 104/208] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:46 2022] GET /identity/v3/role_assignments?user.id=e58d91e284ee49b7b6cf95625c3c9dc2&scope.project.id=32e961e117e24576acf2592d4ee51e41&role.id=eeefde0ca97549ed9cd98300215a29ad => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:47.795271 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5aa8a2-7f8a-4053-886b-d820e04f0c0e None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.796031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5aa8a2-7f8a-4053-886b-d820e04f0c0e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.796031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5aa8a2-7f8a-4053-886b-d820e04f0c0e None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.797545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 105/209] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:47.802465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f9485-8397-4d0f-898c-dcf493aeec73 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.802925 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f9485-8397-4d0f-898c-dcf493aeec73 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.803299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f9485-8397-4d0f-898c-dcf493aeec73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.826462 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-c85f9485-8397-4d0f-898c-dcf493aeec73 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:47.828194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c85f9485-8397-4d0f-898c-dcf493aeec73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:47.903588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 105/210] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:48.108529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38862-5245-4288-94a5-5c7c364e2c44 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.108747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38862-5245-4288-94a5-5c7c364e2c44 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.109181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef38862-5245-4288-94a5-5c7c364e2c44 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.111006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 106/211] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:48.117791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72878f2d-91d0-4265-9f34-3204991f2d33 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.118346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72878f2d-91d0-4265-9f34-3204991f2d33 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.118819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72878f2d-91d0-4265-9f34-3204991f2d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.132966 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.139353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-72878f2d-91d0-4265-9f34-3204991f2d33 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:48.141426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-72878f2d-91d0-4265-9f34-3204991f2d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:48.181854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef38862-5245-4288-94a5-5c7c364e2c44 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.227350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 106/212] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:48.303632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef38862-5245-4288-94a5-5c7c364e2c44 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.312980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08a320-b6eb-4a75-b5f0-e453e0b3623d None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.313561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08a320-b6eb-4a75-b5f0-e453e0b3623d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.314074 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08a320-b6eb-4a75-b5f0-e453e0b3623d None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.316136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 107/213] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:48.323777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c207e-d1c0-4d59-adf2-78651b1fd732 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.324331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c207e-d1c0-4d59-adf2-78651b1fd732 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.324808 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c207e-d1c0-4d59-adf2-78651b1fd732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.344112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-836c207e-d1c0-4d59-adf2-78651b1fd732 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:48.345675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-836c207e-d1c0-4d59-adf2-78651b1fd732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:48.350210 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb912ac-6c79-4a45-aeb5-0863f88de28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.350850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb912ac-6c79-4a45-aeb5-0863f88de28d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.351127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb912ac-6c79-4a45-aeb5-0863f88de28d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.351397 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb912ac-6c79-4a45-aeb5-0863f88de28d None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.352568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb912ac-6c79-4a45-aeb5-0863f88de28d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.354036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb912ac-6c79-4a45-aeb5-0863f88de28d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.355674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 107/214] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:48.361137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb912ac-6c79-4a45-aeb5-0863f88de28d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.380651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.433793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 108/215] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:48.481242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb912ac-6c79-4a45-aeb5-0863f88de28d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.525086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e389d-c6a3-40ec-abe9-25cfa910a138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.525699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e389d-c6a3-40ec-abe9-25cfa910a138 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.526007 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e389d-c6a3-40ec-abe9-25cfa910a138 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.526280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e389d-c6a3-40ec-abe9-25cfa910a138 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.526941 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e389d-c6a3-40ec-abe9-25cfa910a138 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.528390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e389d-c6a3-40ec-abe9-25cfa910a138 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.547483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c207e-d1c0-4d59-adf2-78651b1fd732 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.557005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 108/216] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:48 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:38:48.647222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.697673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c207e-d1c0-4d59-adf2-78651b1fd732 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.744666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59fcbe-484c-4f81-a2cf-a2a2e3cb8729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.745297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59fcbe-484c-4f81-a2cf-a2a2e3cb8729 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.745571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59fcbe-484c-4f81-a2cf-a2a2e3cb8729 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.745878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59fcbe-484c-4f81-a2cf-a2a2e3cb8729 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.757697 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e389d-c6a3-40ec-abe9-25cfa910a138 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.761366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59fcbe-484c-4f81-a2cf-a2a2e3cb8729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-8c59fcbe-484c-4f81-a2cf-a2a2e3cb8729 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:48.766423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:48.767340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:48.768572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:48.768572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:48.768572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:48.768572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:48.768572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:48.768572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:48.768572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:48.768572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:48.768572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:48.768572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:48.768572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:48.768572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:38:48.768572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 109/217] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:48.773531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59fcbe-484c-4f81-a2cf-a2a2e3cb8729 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.851885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.899604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e389d-c6a3-40ec-abe9-25cfa910a138 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.906313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59fcbe-484c-4f81-a2cf-a2a2e3cb8729 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.949941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.956836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cc3ee-33a1-4b44-9b64-3f9154eb82f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.957500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cc3ee-33a1-4b44-9b64-3f9154eb82f8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.957799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cc3ee-33a1-4b44-9b64-3f9154eb82f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.958153 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cc3ee-33a1-4b44-9b64-3f9154eb82f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.966123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd46b8-cb55-4355-8086-adcabce39eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.966740 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd46b8-cb55-4355-8086-adcabce39eb1 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.967015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd46b8-cb55-4355-8086-adcabce39eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.967818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd46b8-cb55-4355-8086-adcabce39eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.968500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd46b8-cb55-4355-8086-adcabce39eb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:48.968797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd46b8-cb55-4355-8086-adcabce39eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.971109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd46b8-cb55-4355-8086-adcabce39eb1 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.974230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557cc3ee-33a1-4b44-9b64-3f9154eb82f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-557cc3ee-33a1-4b44-9b64-3f9154eb82f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:48.979685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:48.980839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:48.982162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:48.982162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:48.982162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:48.982162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:48.982162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:48.982162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:48.982162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:48.982162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:48.982162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:48.982162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:48.982162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:48.982162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:48.983003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 110/218] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:48.983096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 109/219] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:48.986204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cc3ee-33a1-4b44-9b64-3f9154eb82f8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.988005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726f5e0-25bf-4aa7-a8b1-ffcccb8810b6 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.988293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726f5e0-25bf-4aa7-a8b1-ffcccb8810b6 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.988561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726f5e0-25bf-4aa7-a8b1-ffcccb8810b6 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.990683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 111/220] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:48.990683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726f5e0-25bf-4aa7-a8b1-ffcccb8810b6 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.113192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d726f5e0-25bf-4aa7-a8b1-ffcccb8810b6 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.122871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cc3ee-33a1-4b44-9b64-3f9154eb82f8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.156240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69977e0-1929-474e-9900-3a8fc7549756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.157372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69977e0-1929-474e-9900-3a8fc7549756 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.158209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69977e0-1929-474e-9900-3a8fc7549756 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.158698 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69977e0-1929-474e-9900-3a8fc7549756 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.159814 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69977e0-1929-474e-9900-3a8fc7549756 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:49.160321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69977e0-1929-474e-9900-3a8fc7549756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.161129 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.162158 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69977e0-1929-474e-9900-3a8fc7549756 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.171635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fddc9c-768b-410f-8c7f-037bcbddbaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.173183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fddc9c-768b-410f-8c7f-037bcbddbaae None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.174003 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fddc9c-768b-410f-8c7f-037bcbddbaae None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.174460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 112/221] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:49.175167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fddc9c-768b-410f-8c7f-037bcbddbaae None admin] PATH_INFO: `/v3/users/3d8e45af70e0443e89fc38e4234763c7` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.183158 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053db48-5ae9-475c-9778-87bd94527e7a None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.183158 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053db48-5ae9-475c-9778-87bd94527e7a None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.183158 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053db48-5ae9-475c-9778-87bd94527e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.195326 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fddc9c-768b-410f-8c7f-037bcbddbaae None admin] RBAC: Authorizing `identity:get_user(user_id=3d8e45af70e0443e89fc38e4234763c7)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.197185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fddc9c-768b-410f-8c7f-037bcbddbaae None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.199484 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 110/222] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/users/3d8e45af70e0443e89fc38e4234763c7 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:49.200596 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fddc9c-768b-410f-8c7f-037bcbddbaae None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.205207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-2053db48-5ae9-475c-9778-87bd94527e7a None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:49.206497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2053db48-5ae9-475c-9778-87bd94527e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:49.261511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.288806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 113/223] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:49.290255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053db48-5ae9-475c-9778-87bd94527e7a None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.339093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fddc9c-768b-410f-8c7f-037bcbddbaae None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.411414 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305c749-1a03-4fd9-b90a-929a85475514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.414102 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305c749-1a03-4fd9-b90a-929a85475514 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.415757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305c749-1a03-4fd9-b90a-929a85475514 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.416241 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d305c749-1a03-4fd9-b90a-929a85475514 None admin] PATH_INFO: `/v3/users/b6ffdb84fa4a4add84fbcc8070f44a67` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.419901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305c749-1a03-4fd9-b90a-929a85475514 None admin] RBAC: Authorizing `identity:get_user(user_id=b6ffdb84fa4a4add84fbcc8070f44a67)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.422099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d305c749-1a03-4fd9-b90a-929a85475514 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.425208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 111/224] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/users/b6ffdb84fa4a4add84fbcc8070f44a67 => generated 283 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:49.430739 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305c749-1a03-4fd9-b90a-929a85475514 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.488261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053db48-5ae9-475c-9778-87bd94527e7a None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.555882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.563982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e8192-2ec3-4cda-83fd-636aee4a5daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.564607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e8192-2ec3-4cda-83fd-636aee4a5daf None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.565650 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e8192-2ec3-4cda-83fd-636aee4a5daf None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.565957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e8192-2ec3-4cda-83fd-636aee4a5daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.623604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d305c749-1a03-4fd9-b90a-929a85475514 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.631494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.654387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e8192-2ec3-4cda-83fd-636aee4a5daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.656360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e8192-2ec3-4cda-83fd-636aee4a5daf None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.671859 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da55dc-51e0-4189-968a-f89d5b757fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.672964 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da55dc-51e0-4189-968a-f89d5b757fea None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.673395 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da55dc-51e0-4189-968a-f89d5b757fea None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.673868 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da55dc-51e0-4189-968a-f89d5b757fea None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.675879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da55dc-51e0-4189-968a-f89d5b757fea None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.678067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da55dc-51e0-4189-968a-f89d5b757fea None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.680276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 112/225] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:49.686675 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da55dc-51e0-4189-968a-f89d5b757fea None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.783717 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 114/226] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:49.785374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e8192-2ec3-4cda-83fd-636aee4a5daf None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.836683 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da55dc-51e0-4189-968a-f89d5b757fea None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.872006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.883398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.886698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1653e-cc61-4a08-a2c1-e54ece2013a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.887445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1653e-cc61-4a08-a2c1-e54ece2013a9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.888156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1653e-cc61-4a08-a2c1-e54ece2013a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.888546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1653e-cc61-4a08-a2c1-e54ece2013a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.901419 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1653e-cc61-4a08-a2c1-e54ece2013a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-a0e1653e-cc61-4a08-a2c1-e54ece2013a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:49.907951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:49.908879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:49.909695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:49.909695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:49.909695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:49.909695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:49.909695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:49.909695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:49.909695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:49.909695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:49.909695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:49.909695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:49.909695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:49.909695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:49.909695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 113/227] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:49.910517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1653e-cc61-4a08-a2c1-e54ece2013a9 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.921232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e8192-2ec3-4cda-83fd-636aee4a5daf None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.960818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1b027-64cf-46b9-84e7-67564d13a3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.961834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1b027-64cf-46b9-84e7-67564d13a3f0 None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.962338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1b027-64cf-46b9-84e7-67564d13a3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.962811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1b027-64cf-46b9-84e7-67564d13a3f0 None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd/users/b6ffdb84fa4a4add84fbcc8070f44a67/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.965014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1b027-64cf-46b9-84e7-67564d13a3f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b04feb5fcdf4d38baad71c9a87f28fd, user_id=b6ffdb84fa4a4add84fbcc8070f44a67, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.966689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1b027-64cf-46b9-84e7-67564d13a3f0 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.982469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 115/228] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:49 2022] PUT /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd/users/b6ffdb84fa4a4add84fbcc8070f44a67/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:49.988440 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1b027-64cf-46b9-84e7-67564d13a3f0 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.091127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1653e-cc61-4a08-a2c1-e54ece2013a9 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.164864 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0ed45-1101-4aa8-ab5a-fa130b88b04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.165859 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0ed45-1101-4aa8-ab5a-fa130b88b04e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.166207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0ed45-1101-4aa8-ab5a-fa130b88b04e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.166498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0ed45-1101-4aa8-ab5a-fa130b88b04e None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.169089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0ed45-1101-4aa8-ab5a-fa130b88b04e None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.171159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0ed45-1101-4aa8-ab5a-fa130b88b04e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.178762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 114/229] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:50.184603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0ed45-1101-4aa8-ab5a-fa130b88b04e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.207629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1b027-64cf-46b9-84e7-67564d13a3f0 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.252399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffbf54-4e68-4a00-988a-5659a17e375d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.253338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffbf54-4e68-4a00-988a-5659a17e375d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.253819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffbf54-4e68-4a00-988a-5659a17e375d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.254296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffbf54-4e68-4a00-988a-5659a17e375d None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.255394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffbf54-4e68-4a00-988a-5659a17e375d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:50.256254 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffbf54-4e68-4a00-988a-5659a17e375d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.257981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffbf54-4e68-4a00-988a-5659a17e375d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.267202 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:50.270511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 116/230] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:50.278683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffbf54-4e68-4a00-988a-5659a17e375d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.282983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:50.351977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0ed45-1101-4aa8-ab5a-fa130b88b04e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.392817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b426fd1-845e-48d7-a206-e842d55a44ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.393434 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b426fd1-845e-48d7-a206-e842d55a44ab None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.393707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b426fd1-845e-48d7-a206-e842d55a44ab None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.394011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b426fd1-845e-48d7-a206-e842d55a44ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.394738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b426fd1-845e-48d7-a206-e842d55a44ab None admin] RBAC: Adding query filter params (role.id=8abe8932c5974e7db6fb974c588a9d70, scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd, user.id=3d8e45af70e0443e89fc38e4234763c7) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:50.395157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b426fd1-845e-48d7-a206-e842d55a44ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.396670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b426fd1-845e-48d7-a206-e842d55a44ab None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.403297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 115/231] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:50 2022] GET /identity/v3/role_assignments?user.id=3d8e45af70e0443e89fc38e4234763c7&scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd&role.id=8abe8932c5974e7db6fb974c588a9d70 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:50.453774 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffbf54-4e68-4a00-988a-5659a17e375d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.496267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919e92a-f936-4abe-a41c-2a428944d9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.496904 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919e92a-f936-4abe-a41c-2a428944d9bc None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.497185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919e92a-f936-4abe-a41c-2a428944d9bc None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.497459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919e92a-f936-4abe-a41c-2a428944d9bc None admin] PATH_INFO: `/v3/users/b6ffdb84fa4a4add84fbcc8070f44a67` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.498676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919e92a-f936-4abe-a41c-2a428944d9bc None admin] RBAC: Authorizing `identity:get_user(user_id=b6ffdb84fa4a4add84fbcc8070f44a67)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.500222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919e92a-f936-4abe-a41c-2a428944d9bc None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.502223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 117/232] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:50 2022] GET /identity/v3/users/b6ffdb84fa4a4add84fbcc8070f44a67 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:50.508845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b426fd1-845e-48d7-a206-e842d55a44ab None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.628418 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b426fd1-845e-48d7-a206-e842d55a44ab None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.648125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:50.684580 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecc9b7-bea2-4a17-8035-10514f5128cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.685248 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecc9b7-bea2-4a17-8035-10514f5128cf None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.685531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecc9b7-bea2-4a17-8035-10514f5128cf None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.685848 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecc9b7-bea2-4a17-8035-10514f5128cf None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.697758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecc9b7-bea2-4a17-8035-10514f5128cf None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.697758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecc9b7-bea2-4a17-8035-10514f5128cf None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.697758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 116/233] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:50 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:50.699440 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919e92a-f936-4abe-a41c-2a428944d9bc None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.780967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:50.818393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919e92a-f936-4abe-a41c-2a428944d9bc None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.862538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04c97b-6c0b-4549-8fc3-864fa378e1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.863271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04c97b-6c0b-4549-8fc3-864fa378e1e1 None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.863558 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04c97b-6c0b-4549-8fc3-864fa378e1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.863837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04c97b-6c0b-4549-8fc3-864fa378e1e1 None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd/users/b6ffdb84fa4a4add84fbcc8070f44a67/roles/7d0ac07c6114438da47dcfefeb1fe05b` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.866111 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04c97b-6c0b-4549-8fc3-864fa378e1e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b04feb5fcdf4d38baad71c9a87f28fd, user_id=b6ffdb84fa4a4add84fbcc8070f44a67, role_id=7d0ac07c6114438da47dcfefeb1fe05b)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.867690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04c97b-6c0b-4549-8fc3-864fa378e1e1 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.891161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 118/234] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:50 2022] PUT /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd/users/b6ffdb84fa4a4add84fbcc8070f44a67/roles/7d0ac07c6114438da47dcfefeb1fe05b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:51.291874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1735f9-41b8-4894-bd49-c861df1ce31b None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.292386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1735f9-41b8-4894-bd49-c861df1ce31b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.292462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1735f9-41b8-4894-bd49-c861df1ce31b None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:51.293959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 117/235] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:51.300606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b4a6-a250-40ed-92e6-6dd8a8b72e10 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.301003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b4a6-a250-40ed-92e6-6dd8a8b72e10 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.301188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b4a6-a250-40ed-92e6-6dd8a8b72e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:51.321833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-add4b4a6-a250-40ed-92e6-6dd8a8b72e10 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:51.323131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-add4b4a6-a250-40ed-92e6-6dd8a8b72e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:51.400285 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 119/236] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:51.439936 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1daa3f-ec21-4ee5-8b7f-0aa7f2be6e08 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.440215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1daa3f-ec21-4ee5-8b7f-0aa7f2be6e08 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.440491 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1daa3f-ec21-4ee5-8b7f-0aa7f2be6e08 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:51.442141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 118/237] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:51.447770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5476da0-c775-4a86-b964-4f00149d73a0 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.448058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5476da0-c775-4a86-b964-4f00149d73a0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.448333 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5476da0-c775-4a86-b964-4f00149d73a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:51.478113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-a5476da0-c775-4a86-b964-4f00149d73a0 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:51.479304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a5476da0-c775-4a86-b964-4f00149d73a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:51.562925 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:51.578544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 120/238] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:51.805765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1daa3f-ec21-4ee5-8b7f-0aa7f2be6e08 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:51.927136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1daa3f-ec21-4ee5-8b7f-0aa7f2be6e08 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:51.927512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5476da0-c775-4a86-b964-4f00149d73a0 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:51.990281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd418e-efa6-4ed3-964e-09f3095bb860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:51.993124 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd418e-efa6-4ed3-964e-09f3095bb860 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.993611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd418e-efa6-4ed3-964e-09f3095bb860 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.994114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfd418e-efa6-4ed3-964e-09f3095bb860 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.016255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfd418e-efa6-4ed3-964e-09f3095bb860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-dcfd418e-efa6-4ed3-964e-09f3095bb860 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:52.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:52.028969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:52.030361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:52.030361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:52.030361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:52.030361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:52.030361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:52.030361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:52.030361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:52.030361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:52.030361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:52.030361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:52.030361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:38:52.030361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:52.034019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 119/239] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:52.043505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd418e-efa6-4ed3-964e-09f3095bb860 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.116236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5476da0-c775-4a86-b964-4f00149d73a0 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.157724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd71f6-ff68-47e8-afef-c9e07638b802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.158839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd71f6-ff68-47e8-afef-c9e07638b802 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.158951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:52.159449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd71f6-ff68-47e8-afef-c9e07638b802 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.159920 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd71f6-ff68-47e8-afef-c9e07638b802 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.161368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd71f6-ff68-47e8-afef-c9e07638b802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.163147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd71f6-ff68-47e8-afef-c9e07638b802 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.165263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 121/240] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:52.175716 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd71f6-ff68-47e8-afef-c9e07638b802 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.239368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:52.286583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd71f6-ff68-47e8-afef-c9e07638b802 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.288517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfd418e-efa6-4ed3-964e-09f3095bb860 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.332491 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb9097-1127-4d15-a2aa-d849475d1a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.332491 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb9097-1127-4d15-a2aa-d849475d1a70 None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.332491 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb9097-1127-4d15-a2aa-d849475d1a70 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.332491 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb9097-1127-4d15-a2aa-d849475d1a70 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.333215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb9097-1127-4d15-a2aa-d849475d1a70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.335747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb9097-1127-4d15-a2aa-d849475d1a70 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.362495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2132880-9b1b-486d-9cd8-a56b9cd32f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.363154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2132880-9b1b-486d-9cd8-a56b9cd32f15 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.363396 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2132880-9b1b-486d-9cd8-a56b9cd32f15 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.363668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2132880-9b1b-486d-9cd8-a56b9cd32f15 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.364347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2132880-9b1b-486d-9cd8-a56b9cd32f15 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:52.364650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2132880-9b1b-486d-9cd8-a56b9cd32f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.366047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2132880-9b1b-486d-9cd8-a56b9cd32f15 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.375407 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 120/241] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:52.379253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 122/242] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:52 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:38:52.380956 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2132880-9b1b-486d-9cd8-a56b9cd32f15 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.543520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2132880-9b1b-486d-9cd8-a56b9cd32f15 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.558799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:52.595184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4155d43-d445-4cd1-a387-278bfb3bedf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.595834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4155d43-d445-4cd1-a387-278bfb3bedf0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.596137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4155d43-d445-4cd1-a387-278bfb3bedf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.596408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4155d43-d445-4cd1-a387-278bfb3bedf0 None admin] PATH_INFO: `/v3/users/5e721f586d8e4e75a746bc466b033974` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.602398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f89c6-a86e-493d-a587-e2691f47449d None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.602680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f89c6-a86e-493d-a587-e2691f47449d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.602947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f89c6-a86e-493d-a587-e2691f47449d None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.604305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 123/243] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:52.624205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b98b25-803b-4460-880a-e623552e37e9 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.624481 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b98b25-803b-4460-880a-e623552e37e9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.624753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b98b25-803b-4460-880a-e623552e37e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.626682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4155d43-d445-4cd1-a387-278bfb3bedf0 None admin] RBAC: Authorizing `identity:get_user(user_id=5e721f586d8e4e75a746bc466b033974)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.628201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4155d43-d445-4cd1-a387-278bfb3bedf0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.630179 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 121/244] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/users/5e721f586d8e4e75a746bc466b033974 => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:52.642928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4155d43-d445-4cd1-a387-278bfb3bedf0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.653256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-98b98b25-803b-4460-880a-e623552e37e9 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:52.655348 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-98b98b25-803b-4460-880a-e623552e37e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:52.743157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 124/245] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:52.791674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4155d43-d445-4cd1-a387-278bfb3bedf0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.848961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99cb8f-7853-4b63-b852-d1fd69f4c1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.849514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99cb8f-7853-4b63-b852-d1fd69f4c1e6 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.849814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99cb8f-7853-4b63-b852-d1fd69f4c1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.850087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b99cb8f-7853-4b63-b852-d1fd69f4c1e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.909401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:52.960139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99cb8f-7853-4b63-b852-d1fd69f4c1e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.961628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b99cb8f-7853-4b63-b852-d1fd69f4c1e6 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.012759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b98b25-803b-4460-880a-e623552e37e9 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.084506 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.096484 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 122/246] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:38:53.101670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99cb8f-7853-4b63-b852-d1fd69f4c1e6 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.158856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b98b25-803b-4460-880a-e623552e37e9 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.176726 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.199271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c4d73-cfcb-4924-b9cb-90b319bd10ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.199975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c4d73-cfcb-4924-b9cb-90b319bd10ff None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.200359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c4d73-cfcb-4924-b9cb-90b319bd10ff None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.201012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c4d73-cfcb-4924-b9cb-90b319bd10ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.213570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c4d73-cfcb-4924-b9cb-90b319bd10ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-d70c4d73-cfcb-4924-b9cb-90b319bd10ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:53.219626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:53.220696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:53.221503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:53.221503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:53.221503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:53.221503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:53.221503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:53.221503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:53.221503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:53.221503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:53.221503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:53.221503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:53.221503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:53.221503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:38:53.221503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 125/247] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:53.224841 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b99cb8f-7853-4b63-b852-d1fd69f4c1e6 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.226717 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c4d73-cfcb-4924-b9cb-90b319bd10ff None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.279926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6684d46-29d8-4cda-bd85-0cb59e21adb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.281531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6684d46-29d8-4cda-bd85-0cb59e21adb6 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.281865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6684d46-29d8-4cda-bd85-0cb59e21adb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.282146 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6684d46-29d8-4cda-bd85-0cb59e21adb6 None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.284040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6684d46-29d8-4cda-bd85-0cb59e21adb6 None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.285933 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6684d46-29d8-4cda-bd85-0cb59e21adb6 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.287903 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 123/248] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:53.294074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6684d46-29d8-4cda-bd85-0cb59e21adb6 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.374636 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c4d73-cfcb-4924-b9cb-90b319bd10ff None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.378035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.419223 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6684d46-29d8-4cda-bd85-0cb59e21adb6 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.419741 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3358d08-cb26-419a-a865-35650177f7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.420452 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3358d08-cb26-419a-a865-35650177f7d0 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.420943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3358d08-cb26-419a-a865-35650177f7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.421432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3358d08-cb26-419a-a865-35650177f7d0 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.423166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3358d08-cb26-419a-a865-35650177f7d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:53.425084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3358d08-cb26-419a-a865-35650177f7d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.426606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3358d08-cb26-419a-a865-35650177f7d0 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.440896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 126/249] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:53.450883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3358d08-cb26-419a-a865-35650177f7d0 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.500444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea128c-4b93-484d-b3af-7b1ceadc20e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.501061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea128c-4b93-484d-b3af-7b1ceadc20e5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.501340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea128c-4b93-484d-b3af-7b1ceadc20e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.501614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea128c-4b93-484d-b3af-7b1ceadc20e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.502466 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea128c-4b93-484d-b3af-7b1ceadc20e5 None admin] RBAC: Adding query filter params (role.id=eeefde0ca97549ed9cd98300215a29ad, scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd, user.id=5e721f586d8e4e75a746bc466b033974) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:53.502775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea128c-4b93-484d-b3af-7b1ceadc20e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.504436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea128c-4b93-484d-b3af-7b1ceadc20e5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.507431 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.525844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 124/250] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/role_assignments?user.id=5e721f586d8e4e75a746bc466b033974&scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd&role.id=eeefde0ca97549ed9cd98300215a29ad => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:53.527382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad31d61-6a00-43be-9289-ef37d1f44741 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.527668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad31d61-6a00-43be-9289-ef37d1f44741 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.527937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad31d61-6a00-43be-9289-ef37d1f44741 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.529277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 125/251] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:53.535152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f415df-7a83-4583-95cf-330b5d6d1225 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.535425 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f415df-7a83-4583-95cf-330b5d6d1225 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.535698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f415df-7a83-4583-95cf-330b5d6d1225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.576447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-41f415df-7a83-4583-95cf-330b5d6d1225 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:53.578104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-41f415df-7a83-4583-95cf-330b5d6d1225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:53.680319 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3358d08-cb26-419a-a865-35650177f7d0 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.722329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 126/252] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:53.767078 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d07266-6abd-40fd-9b23-876d255f4095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.767745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d07266-6abd-40fd-9b23-876d255f4095 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.768026 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d07266-6abd-40fd-9b23-876d255f4095 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.768303 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d07266-6abd-40fd-9b23-876d255f4095 None admin] PATH_INFO: `/v3/users/b6ffdb84fa4a4add84fbcc8070f44a67` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.769513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d07266-6abd-40fd-9b23-876d255f4095 None admin] RBAC: Authorizing `identity:get_user(user_id=b6ffdb84fa4a4add84fbcc8070f44a67)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.771020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d07266-6abd-40fd-9b23-876d255f4095 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.772984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 127/253] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/users/b6ffdb84fa4a4add84fbcc8070f44a67 => generated 283 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:53.778435 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f415df-7a83-4583-95cf-330b5d6d1225 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.867987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc6be4-864b-4c04-8834-eb64d4e91874 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.868304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc6be4-864b-4c04-8834-eb64d4e91874 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.868540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc6be4-864b-4c04-8834-eb64d4e91874 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.870135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 128/254] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:53.876659 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866be09-6c7a-410c-ad40-5fe4372829df None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.876940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866be09-6c7a-410c-ad40-5fe4372829df None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.877217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f866be09-6c7a-410c-ad40-5fe4372829df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.934841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-f866be09-6c7a-410c-ad40-5fe4372829df None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:53.936514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f866be09-6c7a-410c-ad40-5fe4372829df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:53.949536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f415df-7a83-4583-95cf-330b5d6d1225 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.986606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.999572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.026476 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d68895-9529-4b0e-92c0-c814a0fc5d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.027115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d68895-9529-4b0e-92c0-c814a0fc5d3a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.027389 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d68895-9529-4b0e-92c0-c814a0fc5d3a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.027656 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d68895-9529-4b0e-92c0-c814a0fc5d3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.062435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 129/255] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:54.108641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d68895-9529-4b0e-92c0-c814a0fc5d3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.110148 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d68895-9529-4b0e-92c0-c814a0fc5d3a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.220483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 127/256] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:54.226460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866be09-6c7a-410c-ad40-5fe4372829df None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.328349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d68895-9529-4b0e-92c0-c814a0fc5d3a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.362203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f866be09-6c7a-410c-ad40-5fe4372829df None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.384068 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.423609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a716b26-abe0-4d3a-a90d-e6c7f297a783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.424280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a716b26-abe0-4d3a-a90d-e6c7f297a783 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.424528 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a716b26-abe0-4d3a-a90d-e6c7f297a783 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.424798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a716b26-abe0-4d3a-a90d-e6c7f297a783 None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.427333 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a716b26-abe0-4d3a-a90d-e6c7f297a783 None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.429072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a716b26-abe0-4d3a-a90d-e6c7f297a783 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.430743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 130/257] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:54.436975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a716b26-abe0-4d3a-a90d-e6c7f297a783 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.498982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.508322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d68895-9529-4b0e-92c0-c814a0fc5d3a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.547498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a40f1-a499-429a-b300-64a27eb7d7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.548466 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a40f1-a499-429a-b300-64a27eb7d7d6 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.548954 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a40f1-a499-429a-b300-64a27eb7d7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.549406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a40f1-a499-429a-b300-64a27eb7d7d6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.560416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a40f1-a499-429a-b300-64a27eb7d7d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-7f5a40f1-a499-429a-b300-64a27eb7d7d6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:54.568538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:54.569436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:54.570536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:54.570536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:54.570536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:54.570536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:54.570536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:54.570536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:54.570536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:54.570536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:54.570536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:54.570536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:54.570536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:54.570536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:54.579354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 128/258] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:54.581179 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a40f1-a499-429a-b300-64a27eb7d7d6 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.621169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a716b26-abe0-4d3a-a90d-e6c7f297a783 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.690482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393a98a-66a0-4acd-96c5-60ba7d276524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.691097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393a98a-66a0-4acd-96c5-60ba7d276524 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.691372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393a98a-66a0-4acd-96c5-60ba7d276524 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.691641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393a98a-66a0-4acd-96c5-60ba7d276524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.692373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393a98a-66a0-4acd-96c5-60ba7d276524 None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd, user.id=b6ffdb84fa4a4add84fbcc8070f44a67) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:54.692677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393a98a-66a0-4acd-96c5-60ba7d276524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.695467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393a98a-66a0-4acd-96c5-60ba7d276524 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.711459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 131/259] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/role_assignments?user.id=b6ffdb84fa4a4add84fbcc8070f44a67&scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:54.712283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393a98a-66a0-4acd-96c5-60ba7d276524 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.732701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.758204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a40f1-a499-429a-b300-64a27eb7d7d6 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.815998 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0226360-ce1f-41e8-b161-c53055c171ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.816631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0226360-ce1f-41e8-b161-c53055c171ee None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.816907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0226360-ce1f-41e8-b161-c53055c171ee None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.817215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0226360-ce1f-41e8-b161-c53055c171ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.840102 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0226360-ce1f-41e8-b161-c53055c171ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-c0226360-ce1f-41e8-b161-c53055c171ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:54.845448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:54.846342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:54.847723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:54.847723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:54.847723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:54.847723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:54.847723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:54.847723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:54.847723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:54.847723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:54.847723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:54.847723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:54.847723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:54.847723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:54.847723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 129/260] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:54.852785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0226360-ce1f-41e8-b161-c53055c171ee None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.861017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.867099 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393a98a-66a0-4acd-96c5-60ba7d276524 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.923130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986de98a-fa55-488a-b0fb-5fb5ffe1f8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.924115 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986de98a-fa55-488a-b0fb-5fb5ffe1f8cd None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.924619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986de98a-fa55-488a-b0fb-5fb5ffe1f8cd None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.925091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986de98a-fa55-488a-b0fb-5fb5ffe1f8cd None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.926082 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986de98a-fa55-488a-b0fb-5fb5ffe1f8cd None admin] RBAC: Adding query filter params (name=member) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:54.926657 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986de98a-fa55-488a-b0fb-5fb5ffe1f8cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.928322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986de98a-fa55-488a-b0fb-5fb5ffe1f8cd None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.944191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 132/261] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:54.949853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986de98a-fa55-488a-b0fb-5fb5ffe1f8cd None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.991106 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0226360-ce1f-41e8-b161-c53055c171ee None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.054882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ebb53-eea8-4dee-8be5-11b27b379909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.055416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ebb53-eea8-4dee-8be5-11b27b379909 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.055691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ebb53-eea8-4dee-8be5-11b27b379909 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.055962 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6ebb53-eea8-4dee-8be5-11b27b379909 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.056729 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ebb53-eea8-4dee-8be5-11b27b379909 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:55.057033 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ebb53-eea8-4dee-8be5-11b27b379909 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.067070 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6ebb53-eea8-4dee-8be5-11b27b379909 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.074941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986de98a-fa55-488a-b0fb-5fb5ffe1f8cd None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.079541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 130/262] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:55.085369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ebb53-eea8-4dee-8be5-11b27b379909 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.107148 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:55.121538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f472f-78cc-4e25-a67d-9e1f9cb570c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.122627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f472f-78cc-4e25-a67d-9e1f9cb570c7 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.123127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f472f-78cc-4e25-a67d-9e1f9cb570c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.124168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f472f-78cc-4e25-a67d-9e1f9cb570c7 None admin] PATH_INFO: `/v3/users/3d8e45af70e0443e89fc38e4234763c7` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.125640 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f472f-78cc-4e25-a67d-9e1f9cb570c7 None admin] RBAC: Authorizing `identity:get_user(user_id=3d8e45af70e0443e89fc38e4234763c7)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.127403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17f472f-78cc-4e25-a67d-9e1f9cb570c7 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.129278 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 133/263] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/users/3d8e45af70e0443e89fc38e4234763c7 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:55.135425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f472f-78cc-4e25-a67d-9e1f9cb570c7 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.172761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:55.242989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6ebb53-eea8-4dee-8be5-11b27b379909 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.258749 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f472f-78cc-4e25-a67d-9e1f9cb570c7 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.283169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a2bf0-23b8-4538-bbf3-34c07ebfd3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.284038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a2bf0-23b8-4538-bbf3-34c07ebfd3e2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.284582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a2bf0-23b8-4538-bbf3-34c07ebfd3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.285065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a2bf0-23b8-4538-bbf3-34c07ebfd3e2 None admin] PATH_INFO: `/v3/users/b6ffdb84fa4a4add84fbcc8070f44a67` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.286494 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a2bf0-23b8-4538-bbf3-34c07ebfd3e2 None admin] RBAC: Authorizing `identity:get_user(user_id=b6ffdb84fa4a4add84fbcc8070f44a67)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.288431 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a2bf0-23b8-4538-bbf3-34c07ebfd3e2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.290663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 131/264] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/users/b6ffdb84fa4a4add84fbcc8070f44a67 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:55.297177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a2bf0-23b8-4538-bbf3-34c07ebfd3e2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.299957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b8666-4adf-4322-b60d-774696576902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.300593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b8666-4adf-4322-b60d-774696576902 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.300878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b8666-4adf-4322-b60d-774696576902 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.301154 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b8666-4adf-4322-b60d-774696576902 None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.302589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b8666-4adf-4322-b60d-774696576902 None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.304145 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b8666-4adf-4322-b60d-774696576902 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.305762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 134/265] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:55.311456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b8666-4adf-4322-b60d-774696576902 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.395901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:55.418447 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:55.419817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a2bf0-23b8-4538-bbf3-34c07ebfd3e2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.426718 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b8666-4adf-4322-b60d-774696576902 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.486596 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b86f5-8121-42d4-b69a-93229368f1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.488011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b86f5-8121-42d4-b69a-93229368f1fa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.488929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b86f5-8121-42d4-b69a-93229368f1fa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.490674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b86f5-8121-42d4-b69a-93229368f1fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.491962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6af3ec-f4fa-4034-9aad-9425453806db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.493158 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6af3ec-f4fa-4034-9aad-9425453806db None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.494577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6af3ec-f4fa-4034-9aad-9425453806db None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.495051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6af3ec-f4fa-4034-9aad-9425453806db None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd/users/3d8e45af70e0443e89fc38e4234763c7/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.503183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6af3ec-f4fa-4034-9aad-9425453806db None admin] RBAC: Authorizing `identity:create_grant(project_id=9b04feb5fcdf4d38baad71c9a87f28fd, user_id=3d8e45af70e0443e89fc38e4234763c7, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.504923 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6af3ec-f4fa-4034-9aad-9425453806db None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.523670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 135/266] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:55 2022] PUT /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd/users/3d8e45af70e0443e89fc38e4234763c7/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:55.591001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b86f5-8121-42d4-b69a-93229368f1fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.592563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3b86f5-8121-42d4-b69a-93229368f1fa None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.680618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:55.707040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 132/267] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:55.712748 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6af3ec-f4fa-4034-9aad-9425453806db None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.782322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6548311-5b89-43a1-82f7-c7f24bb715e8 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.782652 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6548311-5b89-43a1-82f7-c7f24bb715e8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.782875 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6548311-5b89-43a1-82f7-c7f24bb715e8 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.784567 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 133/268] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:55.790935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56897f25-c775-4fbe-8b80-5287665836b7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.791217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56897f25-c775-4fbe-8b80-5287665836b7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.791480 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56897f25-c775-4fbe-8b80-5287665836b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.814364 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-56897f25-c775-4fbe-8b80-5287665836b7 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:55.815652 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-56897f25-c775-4fbe-8b80-5287665836b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:55.888221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6af3ec-f4fa-4034-9aad-9425453806db None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.918872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 134/269] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:55.933947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2972e-c8c2-4401-9bc2-3d1c68ca67c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.934802 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2972e-c8c2-4401-9bc2-3d1c68ca67c0 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.935228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2972e-c8c2-4401-9bc2-3d1c68ca67c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.935676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2972e-c8c2-4401-9bc2-3d1c68ca67c0 None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.936946 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2972e-c8c2-4401-9bc2-3d1c68ca67c0 None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.938546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2972e-c8c2-4401-9bc2-3d1c68ca67c0 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.940630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 136/270] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:55.946470 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56897f25-c775-4fbe-8b80-5287665836b7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.070975 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56897f25-c775-4fbe-8b80-5287665836b7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.088627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:56.127150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4478e-973f-49e3-9547-ad5c03db34bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.127773 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4478e-973f-49e3-9547-ad5c03db34bf None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.128051 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4478e-973f-49e3-9547-ad5c03db34bf None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.128322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4478e-973f-49e3-9547-ad5c03db34bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.129063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4478e-973f-49e3-9547-ad5c03db34bf None admin] RBAC: Adding query filter params (role.id=7d0ac07c6114438da47dcfefeb1fe05b, scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd, user.id=b6ffdb84fa4a4add84fbcc8070f44a67) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:56.129367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4478e-973f-49e3-9547-ad5c03db34bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:56.130848 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4478e-973f-49e3-9547-ad5c03db34bf None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:56.142776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 135/271] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/role_assignments?user.id=b6ffdb84fa4a4add84fbcc8070f44a67&scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd&role.id=7d0ac07c6114438da47dcfefeb1fe05b => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:56.231772 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5532018-452f-4b52-987d-c9013a8507ef None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.232096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5532018-452f-4b52-987d-c9013a8507ef None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.232326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5532018-452f-4b52-987d-c9013a8507ef None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.234242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 137/272] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:56.239933 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff34e5c-2e39-4ce1-a6b8-fa03ce9c2b76 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.240220 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff34e5c-2e39-4ce1-a6b8-fa03ce9c2b76 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.240492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff34e5c-2e39-4ce1-a6b8-fa03ce9c2b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.266903 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2ff34e5c-2e39-4ce1-a6b8-fa03ce9c2b76 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:56.268130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2ff34e5c-2e39-4ce1-a6b8-fa03ce9c2b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:56.282902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5532018-452f-4b52-987d-c9013a8507ef None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.332983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:56.355108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 136/273] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:56.474352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5532018-452f-4b52-987d-c9013a8507ef None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.512094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdf28e-567b-4887-8ab8-1c01b6dcc225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.512723 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdf28e-567b-4887-8ab8-1c01b6dcc225 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.512999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdf28e-567b-4887-8ab8-1c01b6dcc225 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.513269 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdf28e-567b-4887-8ab8-1c01b6dcc225 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.527021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdf28e-567b-4887-8ab8-1c01b6dcc225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-30cdf28e-567b-4887-8ab8-1c01b6dcc225 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:56.532630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:56.534788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:56.535870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:56.535870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:56.535870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:56.535870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:56.535870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:56.535870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:56.535870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:56.535870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:56.535870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:56.535870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:56.535870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:56.535870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:38:56.535870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 138/274] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:56.547950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff34e5c-2e39-4ce1-a6b8-fa03ce9c2b76 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.680412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff34e5c-2e39-4ce1-a6b8-fa03ce9c2b76 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.681966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:56.703641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13727e4c-aacc-45f8-8edb-11ff896844cc None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.703928 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13727e4c-aacc-45f8-8edb-11ff896844cc None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.704770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13727e4c-aacc-45f8-8edb-11ff896844cc None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.706444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 139/275] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:56.711842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01449f-8a3b-4b98-92ce-cf29f3df74d8 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.712203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01449f-8a3b-4b98-92ce-cf29f3df74d8 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.712548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01449f-8a3b-4b98-92ce-cf29f3df74d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.726110 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64471463-87ad-40df-b920-31ddd7950a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.726977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64471463-87ad-40df-b920-31ddd7950a4c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.727425 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64471463-87ad-40df-b920-31ddd7950a4c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.727958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64471463-87ad-40df-b920-31ddd7950a4c None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.728794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64471463-87ad-40df-b920-31ddd7950a4c None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:56.729266 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64471463-87ad-40df-b920-31ddd7950a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:56.730933 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64471463-87ad-40df-b920-31ddd7950a4c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:56.736548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-8f01449f-8a3b-4b98-92ce-cf29f3df74d8 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:56.737769 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8f01449f-8a3b-4b98-92ce-cf29f3df74d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:56.740474 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 137/276] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:56.741402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64471463-87ad-40df-b920-31ddd7950a4c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.811056 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 140/277] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:56.812234 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01449f-8a3b-4b98-92ce-cf29f3df74d8 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.834883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:56.859091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64471463-87ad-40df-b920-31ddd7950a4c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.894427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6abc4-9d8f-49c1-ae03-ed07d9e7537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.895130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6abc4-9d8f-49c1-ae03-ed07d9e7537c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.895468 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6abc4-9d8f-49c1-ae03-ed07d9e7537c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.895805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6abc4-9d8f-49c1-ae03-ed07d9e7537c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.897287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6abc4-9d8f-49c1-ae03-ed07d9e7537c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:56.898712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6abc4-9d8f-49c1-ae03-ed07d9e7537c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:56.905170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 138/278] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:56.906555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6abc4-9d8f-49c1-ae03-ed07d9e7537c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.947039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:56.961376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01449f-8a3b-4b98-92ce-cf29f3df74d8 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:57.000010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed510683-baca-4340-9c65-f80cc153044c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:57.000653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed510683-baca-4340-9c65-f80cc153044c None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.000996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed510683-baca-4340-9c65-f80cc153044c None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.001358 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed510683-baca-4340-9c65-f80cc153044c None admin] PATH_INFO: `/v3/users/ebcc53c1f5bb45aca79b43dee537160c` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.016866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed510683-baca-4340-9c65-f80cc153044c None admin] RBAC: Authorizing `identity:get_user(user_id=ebcc53c1f5bb45aca79b43dee537160c)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:57.018880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed510683-baca-4340-9c65-f80cc153044c None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:57.025772 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 141/279] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:56 2022] GET /identity/v3/users/ebcc53c1f5bb45aca79b43dee537160c => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:57.026968 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed510683-baca-4340-9c65-f80cc153044c None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:57.040486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6abc4-9d8f-49c1-ae03-ed07d9e7537c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:57.085245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e68cc-c8dd-40d3-9090-9ccbf8c4203c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:57.086501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e68cc-c8dd-40d3-9090-9ccbf8c4203c None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.086976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e68cc-c8dd-40d3-9090-9ccbf8c4203c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.087298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e68cc-c8dd-40d3-9090-9ccbf8c4203c None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.088212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e68cc-c8dd-40d3-9090-9ccbf8c4203c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:57.090440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e68cc-c8dd-40d3-9090-9ccbf8c4203c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:57.104953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:57.121759 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 139/280] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:56 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:57.121759 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e68cc-c8dd-40d3-9090-9ccbf8c4203c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:57.140478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed510683-baca-4340-9c65-f80cc153044c None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:57.182496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481c32e-fb4a-4b85-99f9-d44458c484f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:57.183118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481c32e-fb4a-4b85-99f9-d44458c484f5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.183398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481c32e-fb4a-4b85-99f9-d44458c484f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.183672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481c32e-fb4a-4b85-99f9-d44458c484f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.184658 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481c32e-fb4a-4b85-99f9-d44458c484f5 None admin] RBAC: Adding query filter params (role.id=8abe8932c5974e7db6fb974c588a9d70, user.id=ebcc53c1f5bb45aca79b43dee537160c, scope.system=all) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:57.185370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481c32e-fb4a-4b85-99f9-d44458c484f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:57.187820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481c32e-fb4a-4b85-99f9-d44458c484f5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:57.194143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 142/281] 10.222.0.55 () {56 vars in 1308 bytes} [Thu Sep 8 13:38:57 2022] GET /identity/v3/role_assignments?user.id=ebcc53c1f5bb45aca79b43dee537160c&scope.system=all&role.id=8abe8932c5974e7db6fb974c588a9d70 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:57.245894 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:57.262322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e68cc-c8dd-40d3-9090-9ccbf8c4203c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:57.306563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164afd0a-8733-45db-8f9f-185e17956f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:57.307229 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164afd0a-8733-45db-8f9f-185e17956f76 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.307513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164afd0a-8733-45db-8f9f-185e17956f76 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.307792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164afd0a-8733-45db-8f9f-185e17956f76 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.321667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164afd0a-8733-45db-8f9f-185e17956f76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-164afd0a-8733-45db-8f9f-185e17956f76 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:57.327874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:57.328825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:57.329631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:57.329631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:57.329631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:57.329631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:57.329631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:57.329631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:57.329631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:57.329631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:57.329631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:57.329631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:57.329631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:38:57.329631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:38:57.329631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 140/282] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:57.335394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481c32e-fb4a-4b85-99f9-d44458c484f5 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:57.449389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481c32e-fb4a-4b85-99f9-d44458c484f5 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:57.488544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6975dd-fd89-49c0-baf5-3c74f87fb18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:57.489270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6975dd-fd89-49c0-baf5-3c74f87fb18d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.489428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6975dd-fd89-49c0-baf5-3c74f87fb18d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.489695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6975dd-fd89-49c0-baf5-3c74f87fb18d None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.490415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6975dd-fd89-49c0-baf5-3c74f87fb18d None admin] RBAC: Adding query filter params (name=reader) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:57.490720 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6975dd-fd89-49c0-baf5-3c74f87fb18d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:57.492093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6975dd-fd89-49c0-baf5-3c74f87fb18d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:57.493476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:57.503029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 143/283] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:57.508573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164afd0a-8733-45db-8f9f-185e17956f76 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:57.632042 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164afd0a-8733-45db-8f9f-185e17956f76 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:57.667537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6081f1-d684-4e1a-94fd-03222d353dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:57.668184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6081f1-d684-4e1a-94fd-03222d353dab None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.668466 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6081f1-d684-4e1a-94fd-03222d353dab None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.668742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6081f1-d684-4e1a-94fd-03222d353dab None admin] PATH_INFO: `/v3/users/5e721f586d8e4e75a746bc466b033974` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.670506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6081f1-d684-4e1a-94fd-03222d353dab None admin] RBAC: Authorizing `identity:get_user(user_id=5e721f586d8e4e75a746bc466b033974)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:57.672040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6081f1-d684-4e1a-94fd-03222d353dab None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:57.673809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 141/284] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:57 2022] GET /identity/v3/users/5e721f586d8e4e75a746bc466b033974 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:57.679489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6975dd-fd89-49c0-baf5-3c74f87fb18d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:57.791319 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6975dd-fd89-49c0-baf5-3c74f87fb18d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:57.826813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70093f77-8d19-4d41-9013-a61789a1e7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:57.827880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70093f77-8d19-4d41-9013-a61789a1e7ad None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.828156 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70093f77-8d19-4d41-9013-a61789a1e7ad None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.828424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70093f77-8d19-4d41-9013-a61789a1e7ad None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.829569 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70093f77-8d19-4d41-9013-a61789a1e7ad None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:57.831050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70093f77-8d19-4d41-9013-a61789a1e7ad None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:57.832848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 144/285] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:57 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:57.838446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6081f1-d684-4e1a-94fd-03222d353dab None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:57.858906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:57.968600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6081f1-d684-4e1a-94fd-03222d353dab None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.015776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dac716-1e4f-4a05-91f9-82413f499fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:58.016491 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dac716-1e4f-4a05-91f9-82413f499fae None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.016775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dac716-1e4f-4a05-91f9-82413f499fae None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.017047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dac716-1e4f-4a05-91f9-82413f499fae None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd/users/5e721f586d8e4e75a746bc466b033974/roles/eeefde0ca97549ed9cd98300215a29ad` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.018987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dac716-1e4f-4a05-91f9-82413f499fae None admin] RBAC: Authorizing `identity:create_grant(project_id=9b04feb5fcdf4d38baad71c9a87f28fd, user_id=5e721f586d8e4e75a746bc466b033974, role_id=eeefde0ca97549ed9cd98300215a29ad)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:58.020446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dac716-1e4f-4a05-91f9-82413f499fae None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:58.039083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 142/286] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:57 2022] PUT /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd/users/5e721f586d8e4e75a746bc466b033974/roles/eeefde0ca97549ed9cd98300215a29ad => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:58.245615 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33802a-c7ef-44a0-bbfb-97dd88aa2ef9 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.245820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33802a-c7ef-44a0-bbfb-97dd88aa2ef9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.246119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33802a-c7ef-44a0-bbfb-97dd88aa2ef9 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.247572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 145/287] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:58.253367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059c967-bc4d-428a-aba3-af1814e9e09c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.253652 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059c967-bc4d-428a-aba3-af1814e9e09c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.253961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059c967-bc4d-428a-aba3-af1814e9e09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.272955 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d059c967-bc4d-428a-aba3-af1814e9e09c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:58.274216 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d059c967-bc4d-428a-aba3-af1814e9e09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:58.362246 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 143/288] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:58.651602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33802a-c7ef-44a0-bbfb-97dd88aa2ef9 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:58.732109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:58.772941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33802a-c7ef-44a0-bbfb-97dd88aa2ef9 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.807690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc5806-3174-4f44-ac73-df6b1f78d3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:58.808319 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc5806-3174-4f44-ac73-df6b1f78d3e2 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.808604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc5806-3174-4f44-ac73-df6b1f78d3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.808879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc5806-3174-4f44-ac73-df6b1f78d3e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.821196 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc5806-3174-4f44-ac73-df6b1f78d3e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-3adc5806-3174-4f44-ac73-df6b1f78d3e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:58.826120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:58.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:58.828309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:58.828309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:58.828309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:58.828309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:58.828309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:58.828309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:58.828309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:58.828309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:58.828309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:58.828309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:58.828309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:58.828309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:38:58.828309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 146/289] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:58.838827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059c967-bc4d-428a-aba3-af1814e9e09c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:58.847324 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:58.980438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059c967-bc4d-428a-aba3-af1814e9e09c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:59.022393 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde51c1-cb0c-4d9b-a09f-1bd91200e416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.023010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde51c1-cb0c-4d9b-a09f-1bd91200e416 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.023287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde51c1-cb0c-4d9b-a09f-1bd91200e416 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.023558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde51c1-cb0c-4d9b-a09f-1bd91200e416 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.024233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde51c1-cb0c-4d9b-a09f-1bd91200e416 None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:59.024536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde51c1-cb0c-4d9b-a09f-1bd91200e416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.025933 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde51c1-cb0c-4d9b-a09f-1bd91200e416 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.034497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 144/290] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:59.042992 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc5806-3174-4f44-ac73-df6b1f78d3e2 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.201478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc5806-3174-4f44-ac73-df6b1f78d3e2 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:59.224412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:59.236018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22d727-6960-46bf-b473-0c6dc5512be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.236648 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22d727-6960-46bf-b473-0c6dc5512be5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.236966 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22d727-6960-46bf-b473-0c6dc5512be5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.237254 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22d727-6960-46bf-b473-0c6dc5512be5 None admin] PATH_INFO: `/v3/users/3d8e45af70e0443e89fc38e4234763c7` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.242465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22d727-6960-46bf-b473-0c6dc5512be5 None admin] RBAC: Authorizing `identity:get_user(user_id=3d8e45af70e0443e89fc38e4234763c7)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.243926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22d727-6960-46bf-b473-0c6dc5512be5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.245543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 147/291] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:59 2022] GET /identity/v3/users/3d8e45af70e0443e89fc38e4234763c7 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:59.254830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde51c1-cb0c-4d9b-a09f-1bd91200e416 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.353397 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:59.393491 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde51c1-cb0c-4d9b-a09f-1bd91200e416 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:59.430035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c8d0c-eef3-4cc4-ac28-a06518ef5a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.430652 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c8d0c-eef3-4cc4-ac28-a06518ef5a30 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.430982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c8d0c-eef3-4cc4-ac28-a06518ef5a30 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.431264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c8d0c-eef3-4cc4-ac28-a06518ef5a30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.520056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c8d0c-eef3-4cc4-ac28-a06518ef5a30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.521430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c8d0c-eef3-4cc4-ac28-a06518ef5a30 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.597961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:59.622035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483a42d-00f7-4169-b945-f891fbff4b0d None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.622662 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483a42d-00f7-4169-b945-f891fbff4b0d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.623695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483a42d-00f7-4169-b945-f891fbff4b0d None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.625224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 148/292] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:59.631239 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0027ce-590c-45f0-9d6b-26115f964c84 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.631518 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0027ce-590c-45f0-9d6b-26115f964c84 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.631788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0027ce-590c-45f0-9d6b-26115f964c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.643054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 145/293] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:59.650722 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c8d0c-eef3-4cc4-ac28-a06518ef5a30 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.672418 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-0a0027ce-590c-45f0-9d6b-26115f964c84 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:59.673623 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0a0027ce-590c-45f0-9d6b-26115f964c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:59.810517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 149/294] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:59.828454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c8d0c-eef3-4cc4-ac28-a06518ef5a30 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:59.869167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07759f66-c487-4c49-ada9-45508e81d8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.870485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07759f66-c487-4c49-ada9-45508e81d8ad None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.870905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07759f66-c487-4c49-ada9-45508e81d8ad None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.871312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07759f66-c487-4c49-ada9-45508e81d8ad None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.878399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07759f66-c487-4c49-ada9-45508e81d8ad None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.880011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07759f66-c487-4c49-ada9-45508e81d8ad None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.882157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 146/295] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:59 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:59.889175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0027ce-590c-45f0-9d6b-26115f964c84 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.039521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0027ce-590c-45f0-9d6b-26115f964c84 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.045940 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214517b3-b805-42b2-85c2-463c03f5d5b9 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.046397 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214517b3-b805-42b2-85c2-463c03f5d5b9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.046809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214517b3-b805-42b2-85c2-463c03f5d5b9 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.048270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 147/296] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:00.053200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b8043-bb41-4e2c-823d-9867341c0ad4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.053657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b8043-bb41-4e2c-823d-9867341c0ad4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.054151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b8043-bb41-4e2c-823d-9867341c0ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.079046 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-fd3b8043-bb41-4e2c-823d-9867341c0ad4 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:00.081284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fd3b8043-bb41-4e2c-823d-9867341c0ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:00.097127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.117863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26511c2e-d47b-40ff-b7ad-ffc10ba8ae85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.119602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26511c2e-d47b-40ff-b7ad-ffc10ba8ae85 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.123051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26511c2e-d47b-40ff-b7ad-ffc10ba8ae85 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.123499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26511c2e-d47b-40ff-b7ad-ffc10ba8ae85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.124237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26511c2e-d47b-40ff-b7ad-ffc10ba8ae85 None admin] RBAC: Adding query filter params (role.id=8abe8932c5974e7db6fb974c588a9d70, scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd, user.id=3d8e45af70e0443e89fc38e4234763c7) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:00.124542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26511c2e-d47b-40ff-b7ad-ffc10ba8ae85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.126562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26511c2e-d47b-40ff-b7ad-ffc10ba8ae85 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.137061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.137987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 150/297] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:59 2022] GET /identity/v3/role_assignments?user.id=3d8e45af70e0443e89fc38e4234763c7&scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd&role.id=8abe8932c5974e7db6fb974c588a9d70 => generated 608 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:00.139219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26511c2e-d47b-40ff-b7ad-ffc10ba8ae85 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.211778 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 148/298] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 08 13:39:00.346133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26511c2e-d47b-40ff-b7ad-ffc10ba8ae85 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.389760 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09051675-c428-477a-be87-21298d0964cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.390410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09051675-c428-477a-be87-21298d0964cf None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.390690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09051675-c428-477a-be87-21298d0964cf None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.390963 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09051675-c428-477a-be87-21298d0964cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.402945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09051675-c428-477a-be87-21298d0964cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-09051675-c428-477a-be87-21298d0964cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:00.408941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:00.409902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:00.411311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:00.411311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:00.411311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:00.411311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:00.411311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:00.411311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:00.411311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:00.411311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:00.411311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:00.411311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:00.411311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:00.411311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:00.411311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 151/299] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:00.417302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b8043-bb41-4e2c-823d-9867341c0ad4 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.492940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209acacf-6cb9-432c-816a-9c9047242bb3 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.493261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209acacf-6cb9-432c-816a-9c9047242bb3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.493486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209acacf-6cb9-432c-816a-9c9047242bb3 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.495001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 152/300] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:00.500040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53fa99-40af-49a1-99a8-fd24579a89f2 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.500324 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53fa99-40af-49a1-99a8-fd24579a89f2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.500604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53fa99-40af-49a1-99a8-fd24579a89f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.524493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-aa53fa99-40af-49a1-99a8-fd24579a89f2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:00.525690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-aa53fa99-40af-49a1-99a8-fd24579a89f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:00.538111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b8043-bb41-4e2c-823d-9867341c0ad4 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.578178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5081a-4399-453e-95a7-0470cb62595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.578607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5081a-4399-453e-95a7-0470cb62595a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.578880 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5081a-4399-453e-95a7-0470cb62595a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.579148 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5081a-4399-453e-95a7-0470cb62595a None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.579768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.579874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5081a-4399-453e-95a7-0470cb62595a None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:00.580085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5081a-4399-453e-95a7-0470cb62595a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.581603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5081a-4399-453e-95a7-0470cb62595a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.591268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 149/301] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:00.592021 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5081a-4399-453e-95a7-0470cb62595a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.608285 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 153/302] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:39:00.609055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53fa99-40af-49a1-99a8-fd24579a89f2 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.634681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.719046 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5081a-4399-453e-95a7-0470cb62595a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.747435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53fa99-40af-49a1-99a8-fd24579a89f2 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.756777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e9b82-a8a8-46f0-a8d5-672dcc0889fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.757977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e9b82-a8a8-46f0-a8d5-672dcc0889fa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.758509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e9b82-a8a8-46f0-a8d5-672dcc0889fa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.758976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e9b82-a8a8-46f0-a8d5-672dcc0889fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.772425 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e9b82-a8a8-46f0-a8d5-672dcc0889fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-276e9b82-a8a8-46f0-a8d5-672dcc0889fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:00.779607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:00.780511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:00.781330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:00.781330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:00.781330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:00.781330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:00.781330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:00.781330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:00.781330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:00.781330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:00.781330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:00.781330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:00.781330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:39:00.781330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:00.784224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a134ef5-4ab8-440a-8950-1ed983cdeafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.784761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a134ef5-4ab8-440a-8950-1ed983cdeafa None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.785125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 150/303] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:00.785390 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a134ef5-4ab8-440a-8950-1ed983cdeafa None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.785663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a134ef5-4ab8-440a-8950-1ed983cdeafa None admin] PATH_INFO: `/v3/users/ebcc53c1f5bb45aca79b43dee537160c` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.786974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a134ef5-4ab8-440a-8950-1ed983cdeafa None admin] RBAC: Authorizing `identity:get_user(user_id=ebcc53c1f5bb45aca79b43dee537160c)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.788411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a134ef5-4ab8-440a-8950-1ed983cdeafa None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.790262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e9b82-a8a8-46f0-a8d5-672dcc0889fa None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.794641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 154/304] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/users/ebcc53c1f5bb45aca79b43dee537160c => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:00.797544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a134ef5-4ab8-440a-8950-1ed983cdeafa None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.843749 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.889842 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.903490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e9b82-a8a8-46f0-a8d5-672dcc0889fa None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.906878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a134ef5-4ab8-440a-8950-1ed983cdeafa None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.942046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662eadb-ca51-4856-9cf3-b98d7d85538b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.942813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662eadb-ca51-4856-9cf3-b98d7d85538b None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.943186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662eadb-ca51-4856-9cf3-b98d7d85538b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.943461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662eadb-ca51-4856-9cf3-b98d7d85538b None admin] PATH_INFO: `/v3/system/users/ebcc53c1f5bb45aca79b43dee537160c/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.943649 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbea85-d915-48ff-bad1-651b618d55e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.944276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbea85-d915-48ff-bad1-651b618d55e7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.944524 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbea85-d915-48ff-bad1-651b618d55e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.945048 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbea85-d915-48ff-bad1-651b618d55e7 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.945204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662eadb-ca51-4856-9cf3-b98d7d85538b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ebcc53c1f5bb45aca79b43dee537160c, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.945770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbea85-d915-48ff-bad1-651b618d55e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:00.946075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbea85-d915-48ff-bad1-651b618d55e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.946641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662eadb-ca51-4856-9cf3-b98d7d85538b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.947464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbea85-d915-48ff-bad1-651b618d55e7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.956869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 155/305] 10.222.0.55 () {58 vars in 1334 bytes} [Thu Sep 8 13:39:00 2022] PUT /identity/v3/system/users/ebcc53c1f5bb45aca79b43dee537160c/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:00.957848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662eadb-ca51-4856-9cf3-b98d7d85538b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.963280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 151/306] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:00.970846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbea85-d915-48ff-bad1-651b618d55e7 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:01.089503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662eadb-ca51-4856-9cf3-b98d7d85538b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:01.102672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:01.110984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbea85-d915-48ff-bad1-651b618d55e7 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:01.133946 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1948ea5-0c21-4441-947a-5d06072f0310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:01.134587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1948ea5-0c21-4441-947a-5d06072f0310 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.134871 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1948ea5-0c21-4441-947a-5d06072f0310 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.135142 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1948ea5-0c21-4441-947a-5d06072f0310 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.136452 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1948ea5-0c21-4441-947a-5d06072f0310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:01.138271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1948ea5-0c21-4441-947a-5d06072f0310 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:01.140062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 156/307] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:01.144879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1719e4-c29d-4921-9fc5-ebbc279f5029 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.144879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1719e4-c29d-4921-9fc5-ebbc279f5029 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.144879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1719e4-c29d-4921-9fc5-ebbc279f5029 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.144879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 157/308] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:01.146071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1719e4-c29d-4921-9fc5-ebbc279f5029 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:01.157164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57291513-1d27-4621-9228-64bc18e8c78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:01.157785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57291513-1d27-4621-9228-64bc18e8c78d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.158063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57291513-1d27-4621-9228-64bc18e8c78d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.158552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57291513-1d27-4621-9228-64bc18e8c78d None admin] PATH_INFO: `/v3/users/ab021469c4364a4ebbfa032cf55b5b64` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.174138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57291513-1d27-4621-9228-64bc18e8c78d None admin] RBAC: Authorizing `identity:get_user(user_id=ab021469c4364a4ebbfa032cf55b5b64)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:01.175613 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57291513-1d27-4621-9228-64bc18e8c78d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:01.177264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 152/309] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/users/ab021469c4364a4ebbfa032cf55b5b64 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:01.178594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb2338-a1dd-450c-93d8-c4638ef3c5ad None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.178879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb2338-a1dd-450c-93d8-c4638ef3c5ad None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.179150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb2338-a1dd-450c-93d8-c4638ef3c5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.197179 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1bbb2338-a1dd-450c-93d8-c4638ef3c5ad None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:01.198394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1bbb2338-a1dd-450c-93d8-c4638ef3c5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:01.200386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:01.257069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1719e4-c29d-4921-9fc5-ebbc279f5029 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:01.268086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 153/310] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:01.268934 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb2338-a1dd-450c-93d8-c4638ef3c5ad None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:01.297275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c27736-5925-45c7-ac4c-c2f469838f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:01.298259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c27736-5925-45c7-ac4c-c2f469838f95 None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.298794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c27736-5925-45c7-ac4c-c2f469838f95 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.299243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c27736-5925-45c7-ac4c-c2f469838f95 None admin] PATH_INFO: `/v3/groups` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.300047 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c27736-5925-45c7-ac4c-c2f469838f95 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:01.301842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c27736-5925-45c7-ac4c-c2f469838f95 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:01.325342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 158/311] 10.222.0.55 () {60 vars in 1134 bytes} [Thu Sep 8 13:39:01 2022] POST /identity/v3/groups => generated 229 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:01.402989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb2338-a1dd-450c-93d8-c4638ef3c5ad None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:01.443645 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c06c8d-6512-4030-a5ea-2b690e12bd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:01.444331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c06c8d-6512-4030-a5ea-2b690e12bd85 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.444614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c06c8d-6512-4030-a5ea-2b690e12bd85 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.445145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c06c8d-6512-4030-a5ea-2b690e12bd85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.445944 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c06c8d-6512-4030-a5ea-2b690e12bd85 None admin] RBAC: Adding query filter params (role.id=eeefde0ca97549ed9cd98300215a29ad, user.id=ab021469c4364a4ebbfa032cf55b5b64, scope.system=all) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:01.446303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c06c8d-6512-4030-a5ea-2b690e12bd85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:01.447885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c06c8d-6512-4030-a5ea-2b690e12bd85 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:01.454994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 154/312] 10.222.0.55 () {56 vars in 1308 bytes} [Thu Sep 8 13:39:01 2022] GET /identity/v3/role_assignments?user.id=ab021469c4364a4ebbfa032cf55b5b64&scope.system=all&role.id=eeefde0ca97549ed9cd98300215a29ad => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:01.552058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c27736-5925-45c7-ac4c-c2f469838f95 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:01.594477 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:01.668252 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c27736-5925-45c7-ac4c-c2f469838f95 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:01.704198 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc030-86bf-4f1f-8eb1-f32dadae5e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:01.704836 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc030-86bf-4f1f-8eb1-f32dadae5e55 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.705113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc030-86bf-4f1f-8eb1-f32dadae5e55 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.705385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc030-86bf-4f1f-8eb1-f32dadae5e55 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.716421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdc030-86bf-4f1f-8eb1-f32dadae5e55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-69fdc030-86bf-4f1f-8eb1-f32dadae5e55 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:01.721855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:01.722751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:01.723552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:01.723552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:01.723552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:01.723552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:01.723552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:01.723552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:01.723552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:01.723552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:01.723552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:01.723552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:01.723552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:39:01.723552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:01.723552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 159/313] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:01.729092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c06c8d-6512-4030-a5ea-2b690e12bd85 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:01.731921 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:01.836515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c06c8d-6512-4030-a5ea-2b690e12bd85 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:01.873316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e418c-6119-4b84-be22-ed5fb66abaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:01.873976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e418c-6119-4b84-be22-ed5fb66abaa2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.874258 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e418c-6119-4b84-be22-ed5fb66abaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.874527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e418c-6119-4b84-be22-ed5fb66abaa2 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.875286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e418c-6119-4b84-be22-ed5fb66abaa2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:01.875594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e418c-6119-4b84-be22-ed5fb66abaa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:01.877033 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986e418c-6119-4b84-be22-ed5fb66abaa2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:01.887259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 155/314] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:01.892780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc030-86bf-4f1f-8eb1-f32dadae5e55 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:02.008493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc030-86bf-4f1f-8eb1-f32dadae5e55 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:02.018360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:02.043883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf1784-4e9d-41ae-a040-603459387cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:02.044509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf1784-4e9d-41ae-a040-603459387cce None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.045317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf1784-4e9d-41ae-a040-603459387cce None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.045587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf1784-4e9d-41ae-a040-603459387cce None admin] PATH_INFO: `/v3/users/5e721f586d8e4e75a746bc466b033974` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.046750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf1784-4e9d-41ae-a040-603459387cce None admin] RBAC: Authorizing `identity:get_user(user_id=5e721f586d8e4e75a746bc466b033974)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:02.048195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf1784-4e9d-41ae-a040-603459387cce None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:02.049786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 160/315] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:01 2022] GET /identity/v3/users/5e721f586d8e4e75a746bc466b033974 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:02.055270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e418c-6119-4b84-be22-ed5fb66abaa2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:02.121137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:02.161944 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986e418c-6119-4b84-be22-ed5fb66abaa2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:02.197437 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed108d-0c48-4fca-b78d-212478a44ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:02.198089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed108d-0c48-4fca-b78d-212478a44ba7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.198367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed108d-0c48-4fca-b78d-212478a44ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.198641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed108d-0c48-4fca-b78d-212478a44ba7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.270659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed108d-0c48-4fca-b78d-212478a44ba7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:02.272053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed108d-0c48-4fca-b78d-212478a44ba7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:02.338590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:02.378866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 156/316] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:02.384413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf1784-4e9d-41ae-a040-603459387cce None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:02.488368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf1784-4e9d-41ae-a040-603459387cce None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:02.523168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a334756e-3567-433f-9db0-54beb1e363d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:02.523725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334756e-3567-433f-9db0-54beb1e363d4 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.524005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334756e-3567-433f-9db0-54beb1e363d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.524276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334756e-3567-433f-9db0-54beb1e363d4 None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.525475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334756e-3567-433f-9db0-54beb1e363d4 None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:02.526954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a334756e-3567-433f-9db0-54beb1e363d4 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:02.528583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 161/317] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:02 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:02.534972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed108d-0c48-4fca-b78d-212478a44ba7 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:02.642811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed108d-0c48-4fca-b78d-212478a44ba7 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:02.682974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911c0e4-a194-4c84-b11c-b450446a0b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:02.683584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911c0e4-a194-4c84-b11c-b450446a0b6d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.683866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911c0e4-a194-4c84-b11c-b450446a0b6d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.684135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911c0e4-a194-4c84-b11c-b450446a0b6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.684863 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911c0e4-a194-4c84-b11c-b450446a0b6d None admin] RBAC: Adding query filter params (role.id=eeefde0ca97549ed9cd98300215a29ad, scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd, user.id=5e721f586d8e4e75a746bc466b033974) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:02.685168 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911c0e4-a194-4c84-b11c-b450446a0b6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:02.687164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911c0e4-a194-4c84-b11c-b450446a0b6d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:02.693804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 157/318] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:02 2022] GET /identity/v3/role_assignments?user.id=5e721f586d8e4e75a746bc466b033974&scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd&role.id=eeefde0ca97549ed9cd98300215a29ad => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:03.188708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7f3de-a674-4605-98f4-be6e6fdf18cb None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.189214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7f3de-a674-4605-98f4-be6e6fdf18cb None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.189214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7f3de-a674-4605-98f4-be6e6fdf18cb None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.190705 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 162/319] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:03.196208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e582954-bb56-4e5b-9fbf-822c09b9eb4c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.196627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e582954-bb56-4e5b-9fbf-822c09b9eb4c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.196796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e582954-bb56-4e5b-9fbf-822c09b9eb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.214837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4e582954-bb56-4e5b-9fbf-822c09b9eb4c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:03.216006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4e582954-bb56-4e5b-9fbf-822c09b9eb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:03.240654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.285840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 158/320] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:03.557840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7f3de-a674-4605-98f4-be6e6fdf18cb None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.563549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205db254-0fca-4ecd-bf25-04d4f9cbbc35 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.563838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205db254-0fca-4ecd-bf25-04d4f9cbbc35 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.564116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205db254-0fca-4ecd-bf25-04d4f9cbbc35 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.565484 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 159/321] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:03.570714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def634e8-c0e0-4292-9348-4f833982a4ce None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.570992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def634e8-c0e0-4292-9348-4f833982a4ce None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.571259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def634e8-c0e0-4292-9348-4f833982a4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.588788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-def634e8-c0e0-4292-9348-4f833982a4ce None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:03.589971 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-def634e8-c0e0-4292-9348-4f833982a4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:03.594533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.665085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 160/322] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:39:03.673044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7f3de-a674-4605-98f4-be6e6fdf18cb None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.706720 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555cde3-fd3b-4c1e-9346-e86ca484b5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.707343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555cde3-fd3b-4c1e-9346-e86ca484b5f3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.707617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555cde3-fd3b-4c1e-9346-e86ca484b5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.707885 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555cde3-fd3b-4c1e-9346-e86ca484b5f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.718423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555cde3-fd3b-4c1e-9346-e86ca484b5f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-3555cde3-fd3b-4c1e-9346-e86ca484b5f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:03.723204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:03.724089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:03.724932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:03.724932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:03.724932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:03.724932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:03.724932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:03.724932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:03.724932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:03.724932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:03.724932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:03.724932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:03.724932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:03.724932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:03.724932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 163/323] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:03.726364 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18ae5b-bbd6-422b-b7ff-209a557ef6b4 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.726557 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18ae5b-bbd6-422b-b7ff-209a557ef6b4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.726836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18ae5b-bbd6-422b-b7ff-209a557ef6b4 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.728286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 161/324] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:03.731103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555cde3-fd3b-4c1e-9346-e86ca484b5f3 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.732892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7fc50-841a-4317-b12c-6457de7b0f45 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.733173 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7fc50-841a-4317-b12c-6457de7b0f45 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.733446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7fc50-841a-4317-b12c-6457de7b0f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.751097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-01e7fc50-841a-4317-b12c-6457de7b0f45 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:03.752262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-01e7fc50-841a-4317-b12c-6457de7b0f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:03.823143 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 162/325] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:39:03.827833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.837381 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555cde3-fd3b-4c1e-9346-e86ca484b5f3 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.871770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792c9ae-aadd-4de3-8f74-e6a50b824a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.872363 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792c9ae-aadd-4de3-8f74-e6a50b824a2e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.872638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792c9ae-aadd-4de3-8f74-e6a50b824a2e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.873419 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792c9ae-aadd-4de3-8f74-e6a50b824a2e None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.874102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792c9ae-aadd-4de3-8f74-e6a50b824a2e None admin] RBAC: Adding query filter params (name=member) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:03.874401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792c9ae-aadd-4de3-8f74-e6a50b824a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.875746 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792c9ae-aadd-4de3-8f74-e6a50b824a2e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.883895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 164/326] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:03.889886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7fc50-841a-4317-b12c-6457de7b0f45 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.911020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.938561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792c9ae-aadd-4de3-8f74-e6a50b824a2e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.996165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7fc50-841a-4317-b12c-6457de7b0f45 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.031902 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d545af-c521-45e8-9c20-64d901066687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.032546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d545af-c521-45e8-9c20-64d901066687 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.032823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d545af-c521-45e8-9c20-64d901066687 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.033096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d545af-c521-45e8-9c20-64d901066687 None admin] PATH_INFO: `/v3/users/ebcc53c1f5bb45aca79b43dee537160c` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.034257 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d545af-c521-45e8-9c20-64d901066687 None admin] RBAC: Authorizing `identity:get_user(user_id=ebcc53c1f5bb45aca79b43dee537160c)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.035693 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d545af-c521-45e8-9c20-64d901066687 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.037565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 163/327] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/users/ebcc53c1f5bb45aca79b43dee537160c => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:04.043518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d545af-c521-45e8-9c20-64d901066687 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.054391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792c9ae-aadd-4de3-8f74-e6a50b824a2e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.092821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfd952-cabd-4339-9505-87950f8f5518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.093647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfd952-cabd-4339-9505-87950f8f5518 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.093950 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfd952-cabd-4339-9505-87950f8f5518 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.094216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfd952-cabd-4339-9505-87950f8f5518 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.095320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfd952-cabd-4339-9505-87950f8f5518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.096749 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfd952-cabd-4339-9505-87950f8f5518 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.098332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 165/328] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:04.103598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfd952-cabd-4339-9505-87950f8f5518 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.126871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.128428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.142988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d545af-c521-45e8-9c20-64d901066687 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.177183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a1a0b-1269-4177-b3d7-af1898a383d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.177823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a1a0b-1269-4177-b3d7-af1898a383d6 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.178104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a1a0b-1269-4177-b3d7-af1898a383d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.178385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a1a0b-1269-4177-b3d7-af1898a383d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.179109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a1a0b-1269-4177-b3d7-af1898a383d6 None admin] RBAC: Adding query filter params (role.id=8abe8932c5974e7db6fb974c588a9d70, user.id=ebcc53c1f5bb45aca79b43dee537160c, scope.system=all) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:04.179413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a1a0b-1269-4177-b3d7-af1898a383d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.180844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6a1a0b-1269-4177-b3d7-af1898a383d6 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.192200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 164/329] 10.222.0.55 () {56 vars in 1308 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/role_assignments?user.id=ebcc53c1f5bb45aca79b43dee537160c&scope.system=all&role.id=8abe8932c5974e7db6fb974c588a9d70 => generated 510 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:04.192200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a1a0b-1269-4177-b3d7-af1898a383d6 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.210922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfd952-cabd-4339-9505-87950f8f5518 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.250575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d54fa1-4d4e-46d7-848a-6bb4ffe55cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.251425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d54fa1-4d4e-46d7-848a-6bb4ffe55cad None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.251701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d54fa1-4d4e-46d7-848a-6bb4ffe55cad None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.251966 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d54fa1-4d4e-46d7-848a-6bb4ffe55cad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.266697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d54fa1-4d4e-46d7-848a-6bb4ffe55cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-64d54fa1-4d4e-46d7-848a-6bb4ffe55cad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:04.272554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:04.273522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:04.274383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:04.274383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:04.274383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:04.274383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:04.274383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:04.274383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:04.274383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:04.274383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:04.274383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:04.274383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:04.274383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:39:04.274383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:04.274383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 166/330] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:04.280355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d54fa1-4d4e-46d7-848a-6bb4ffe55cad None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.309652 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a1a0b-1269-4177-b3d7-af1898a383d6 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.349148 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbe22d-64c9-4fda-b902-34e7ddb026a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.349677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbe22d-64c9-4fda-b902-34e7ddb026a3 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.349972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbe22d-64c9-4fda-b902-34e7ddb026a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.350235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbe22d-64c9-4fda-b902-34e7ddb026a3 None admin] PATH_INFO: `/v3/groups` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.350883 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbe22d-64c9-4fda-b902-34e7ddb026a3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.352332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbe22d-64c9-4fda-b902-34e7ddb026a3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.369430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 165/331] 10.222.0.55 () {60 vars in 1134 bytes} [Thu Sep 8 13:39:04 2022] POST /identity/v3/groups => generated 226 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:04.381063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.406353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d54fa1-4d4e-46d7-848a-6bb4ffe55cad None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.445854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc65b53-c917-4a09-865e-66c23da8db14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.446485 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc65b53-c917-4a09-865e-66c23da8db14 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.446760 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc65b53-c917-4a09-865e-66c23da8db14 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.447026 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc65b53-c917-4a09-865e-66c23da8db14 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.447769 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc65b53-c917-4a09-865e-66c23da8db14 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:04.448069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc65b53-c917-4a09-865e-66c23da8db14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.449483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc65b53-c917-4a09-865e-66c23da8db14 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.459563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 167/332] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:04.465562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbe22d-64c9-4fda-b902-34e7ddb026a3 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.470000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.570329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbe22d-64c9-4fda-b902-34e7ddb026a3 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.604826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b9373-92de-47b8-aa4d-94ace11a2e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.608006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b9373-92de-47b8-aa4d-94ace11a2e90 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.608006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b9373-92de-47b8-aa4d-94ace11a2e90 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.608006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b9373-92de-47b8-aa4d-94ace11a2e90 None admin] PATH_INFO: `/v3/users/ab021469c4364a4ebbfa032cf55b5b64` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.608006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b9373-92de-47b8-aa4d-94ace11a2e90 None admin] RBAC: Authorizing `identity:get_user(user_id=ab021469c4364a4ebbfa032cf55b5b64)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.608494 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b9373-92de-47b8-aa4d-94ace11a2e90 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.610183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 166/333] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/users/ab021469c4364a4ebbfa032cf55b5b64 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:04.616088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc65b53-c917-4a09-865e-66c23da8db14 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.726251 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc65b53-c917-4a09-865e-66c23da8db14 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.760636 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700fdea-f51c-4501-a3bf-58aef172fedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.761277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700fdea-f51c-4501-a3bf-58aef172fedb None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.761553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700fdea-f51c-4501-a3bf-58aef172fedb None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.761850 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700fdea-f51c-4501-a3bf-58aef172fedb None admin] PATH_INFO: `/v3/system/users/ab021469c4364a4ebbfa032cf55b5b64/roles/eeefde0ca97549ed9cd98300215a29ad` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.763304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700fdea-f51c-4501-a3bf-58aef172fedb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ab021469c4364a4ebbfa032cf55b5b64, role_id=eeefde0ca97549ed9cd98300215a29ad)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.764704 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700fdea-f51c-4501-a3bf-58aef172fedb None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.773784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 168/334] 10.222.0.55 () {58 vars in 1334 bytes} [Thu Sep 8 13:39:04 2022] PUT /identity/v3/system/users/ab021469c4364a4ebbfa032cf55b5b64/roles/eeefde0ca97549ed9cd98300215a29ad => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:06.882024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ccac1-43d9-4ce1-8af6-1e48adbe4145 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:06.882975 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ccac1-43d9-4ce1-8af6-1e48adbe4145 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:06.883473 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ccac1-43d9-4ce1-8af6-1e48adbe4145 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:06.885281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 167/335] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:06.890690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6ecd2-d612-4704-ac83-9c5629d78dd2 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:06.891143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6ecd2-d612-4704-ac83-9c5629d78dd2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:06.891327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6ecd2-d612-4704-ac83-9c5629d78dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:06.917341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-f4e6ecd2-d612-4704-ac83-9c5629d78dd2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:06.918578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f4e6ecd2-d612-4704-ac83-9c5629d78dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:06.920761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:07.012382 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 169/336] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:07.300714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5731b72-6d72-48ec-89ac-471b799c9b62 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.300902 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5731b72-6d72-48ec-89ac-471b799c9b62 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.301162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5731b72-6d72-48ec-89ac-471b799c9b62 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.302836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 168/337] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:07.308442 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db13f6-2b13-48f8-bcf0-e3cddda708b2 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.308730 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db13f6-2b13-48f8-bcf0-e3cddda708b2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.309001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db13f6-2b13-48f8-bcf0-e3cddda708b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.336277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-45db13f6-2b13-48f8-bcf0-e3cddda708b2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:07.337450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-45db13f6-2b13-48f8-bcf0-e3cddda708b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:07.418980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 170/338] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:39:07.586456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b619ce-4cb8-4d5c-ab14-157004d8ab48 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.586634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b619ce-4cb8-4d5c-ab14-157004d8ab48 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.586906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b619ce-4cb8-4d5c-ab14-157004d8ab48 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.588521 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 169/339] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:07.594343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec73e3-697c-4ddd-a813-5bf7a88eff3a None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.594623 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec73e3-697c-4ddd-a813-5bf7a88eff3a None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.594893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dec73e3-697c-4ddd-a813-5bf7a88eff3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.618692 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-3dec73e3-697c-4ddd-a813-5bf7a88eff3a None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:07.619928 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-3dec73e3-697c-4ddd-a813-5bf7a88eff3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:07.623832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b6b93-0ff4-4fa4-84b3-5888d34ee135 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.624143 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b6b93-0ff4-4fa4-84b3-5888d34ee135 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.624414 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b6b93-0ff4-4fa4-84b3-5888d34ee135 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.625764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 170/340] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:07.633296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccec157-51e2-4e92-a036-b0fab5c8b7c4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.633575 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccec157-51e2-4e92-a036-b0fab5c8b7c4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.633910 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccec157-51e2-4e92-a036-b0fab5c8b7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.664486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1ccec157-51e2-4e92-a036-b0fab5c8b7c4 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:07.665655 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1ccec157-51e2-4e92-a036-b0fab5c8b7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:07.754169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:07.761218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 171/341] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:07.763099 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec73e3-697c-4ddd-a813-5bf7a88eff3a None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:07.786611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 171/342] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:07.886145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccec157-51e2-4e92-a036-b0fab5c8b7c4 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:07.886901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dec73e3-697c-4ddd-a813-5bf7a88eff3a None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:07.889722 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:07.925492 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e43a7-a707-42cf-b7b0-4a78e86958b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:07.926159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e43a7-a707-42cf-b7b0-4a78e86958b0 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.926431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e43a7-a707-42cf-b7b0-4a78e86958b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.926695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982e43a7-a707-42cf-b7b0-4a78e86958b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.938414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982e43a7-a707-42cf-b7b0-4a78e86958b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-982e43a7-a707-42cf-b7b0-4a78e86958b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:07.943425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:07.944480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:07.945743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:07.945743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:07.945743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:07.945743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:07.945743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:07.945743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:07.945743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:07.945743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:07.945743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:07.945743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:07.945743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:07.945743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:07.947609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 172/343] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:07.954102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e43a7-a707-42cf-b7b0-4a78e86958b0 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:08.028248 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:08.077588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982e43a7-a707-42cf-b7b0-4a78e86958b0 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:08.100670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccec157-51e2-4e92-a036-b0fab5c8b7c4 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:08.124354 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0358f3-dc7b-43ef-858d-a5784a52447e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:08.125640 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0358f3-dc7b-43ef-858d-a5784a52447e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.126140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0358f3-dc7b-43ef-858d-a5784a52447e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.126561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0358f3-dc7b-43ef-858d-a5784a52447e None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.127407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0358f3-dc7b-43ef-858d-a5784a52447e None admin] RBAC: Adding query filter params (name=member) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:08.127853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0358f3-dc7b-43ef-858d-a5784a52447e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:08.129896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0358f3-dc7b-43ef-858d-a5784a52447e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:08.159657 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 173/344] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:08.165927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9547b1-0d73-488c-85ca-932e7786da4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:08.167626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0358f3-dc7b-43ef-858d-a5784a52447e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:08.168789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9547b1-0d73-488c-85ca-932e7786da4b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.169207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9547b1-0d73-488c-85ca-932e7786da4b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.170166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9547b1-0d73-488c-85ca-932e7786da4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.199594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9547b1-0d73-488c-85ca-932e7786da4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-0b9547b1-0d73-488c-85ca-932e7786da4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:08.210230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:08.211306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:08.212270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:08.212270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:08.212270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:08.212270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:08.212270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:08.212270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:08.212270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:08.212270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:08.212270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:08.212270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:08.212270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:08.212270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:08.216693 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 172/345] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:08.221827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9547b1-0d73-488c-85ca-932e7786da4b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:08.302697 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:08.349659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9547b1-0d73-488c-85ca-932e7786da4b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:08.401936 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d9b55-ea0b-4bfe-a523-bb31a445c954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:08.402826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d9b55-ea0b-4bfe-a523-bb31a445c954 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.403242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d9b55-ea0b-4bfe-a523-bb31a445c954 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.403651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d9b55-ea0b-4bfe-a523-bb31a445c954 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.404438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0358f3-dc7b-43ef-858d-a5784a52447e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:08.405141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d9b55-ea0b-4bfe-a523-bb31a445c954 None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:08.405594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d9b55-ea0b-4bfe-a523-bb31a445c954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:08.407489 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d9b55-ea0b-4bfe-a523-bb31a445c954 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:08.422825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 173/346] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:08.423915 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d9b55-ea0b-4bfe-a523-bb31a445c954 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:08.435836 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:08.456105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6013fc4-c258-482f-9025-7bc9fbc63e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:08.457283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6013fc4-c258-482f-9025-7bc9fbc63e17 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.457762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6013fc4-c258-482f-9025-7bc9fbc63e17 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.458253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6013fc4-c258-482f-9025-7bc9fbc63e17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.542296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6013fc4-c258-482f-9025-7bc9fbc63e17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:08.544599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6013fc4-c258-482f-9025-7bc9fbc63e17 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:08.548265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d9b55-ea0b-4bfe-a523-bb31a445c954 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:08.578152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:08.588907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548f882-59cc-436a-9a54-bfba27dfb6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:08.589548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548f882-59cc-436a-9a54-bfba27dfb6e8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.589854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548f882-59cc-436a-9a54-bfba27dfb6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.590128 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548f882-59cc-436a-9a54-bfba27dfb6e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.604012 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548f882-59cc-436a-9a54-bfba27dfb6e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f548f882-59cc-436a-9a54-bfba27dfb6e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:08.609544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:08.610498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:08.611786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:08.611786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:08.611786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:08.611786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:08.611786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:08.611786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:08.611786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:08.611786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:08.611786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:08.611786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:08.611786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:39:08.611786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:08.611786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 174/347] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:08.612458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548f882-59cc-436a-9a54-bfba27dfb6e8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:08.672778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 174/348] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:39:08.674144 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6013fc4-c258-482f-9025-7bc9fbc63e17 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:08.701786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:08.741833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548f882-59cc-436a-9a54-bfba27dfb6e8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:08.777281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339d43c-7adb-4d0b-a98b-c179f7dc6497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:08.779228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339d43c-7adb-4d0b-a98b-c179f7dc6497 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.779228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339d43c-7adb-4d0b-a98b-c179f7dc6497 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.779228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339d43c-7adb-4d0b-a98b-c179f7dc6497 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.789515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339d43c-7adb-4d0b-a98b-c179f7dc6497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1339d43c-7adb-4d0b-a98b-c179f7dc6497 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:08.795891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:08.796916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:08.797900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:08.797900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:08.797900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:08.797900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:08.797900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:08.797900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:08.797900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:08.797900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:08.797900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:08.797900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:08.797900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:08.797900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:08.798503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6013fc4-c258-482f-9025-7bc9fbc63e17 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:08.800093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 175/349] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:08.801608 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f9d5-a764-4a0e-9d8f-5764f4d48bbf None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.801948 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f9d5-a764-4a0e-9d8f-5764f4d48bbf None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.802220 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42f9d5-a764-4a0e-9d8f-5764f4d48bbf None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.803658 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 176/350] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:08.804247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f9d5-a764-4a0e-9d8f-5764f4d48bbf None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:08.852931 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621db3e-9a06-42ef-a908-2c3767b50741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:08.853936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621db3e-9a06-42ef-a908-2c3767b50741 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.854457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621db3e-9a06-42ef-a908-2c3767b50741 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.854783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621db3e-9a06-42ef-a908-2c3767b50741 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.865894 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:08.948607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42f9d5-a764-4a0e-9d8f-5764f4d48bbf None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:08.950409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621db3e-9a06-42ef-a908-2c3767b50741 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:08.953353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621db3e-9a06-42ef-a908-2c3767b50741 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:08.977407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.001428 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eed859-a610-48f4-9baf-979e98fe1797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.002157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eed859-a610-48f4-9baf-979e98fe1797 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.002459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eed859-a610-48f4-9baf-979e98fe1797 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.002824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eed859-a610-48f4-9baf-979e98fe1797 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.003574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eed859-a610-48f4-9baf-979e98fe1797 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:09.003942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eed859-a610-48f4-9baf-979e98fe1797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:09.005588 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eed859-a610-48f4-9baf-979e98fe1797 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:09.019814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 177/351] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:09.027137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eed859-a610-48f4-9baf-979e98fe1797 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.081547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 175/352] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:09.083151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705529ac-98ec-4230-9e18-aac67f0bfdc1 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.083495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705529ac-98ec-4230-9e18-aac67f0bfdc1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.083834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705529ac-98ec-4230-9e18-aac67f0bfdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.102853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-705529ac-98ec-4230-9e18-aac67f0bfdc1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:09.104144 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-705529ac-98ec-4230-9e18-aac67f0bfdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:09.146365 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eed859-a610-48f4-9baf-979e98fe1797 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.157420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.189603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675173c-a30a-4eba-a64e-5d1919ad84d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.190280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675173c-a30a-4eba-a64e-5d1919ad84d0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.190565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675173c-a30a-4eba-a64e-5d1919ad84d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.190842 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675173c-a30a-4eba-a64e-5d1919ad84d0 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.191980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675173c-a30a-4eba-a64e-5d1919ad84d0 None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:09.193482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675173c-a30a-4eba-a64e-5d1919ad84d0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:09.195636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 178/353] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:09.201204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675173c-a30a-4eba-a64e-5d1919ad84d0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.201574 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 176/354] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:09.202695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705529ac-98ec-4230-9e18-aac67f0bfdc1 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.274167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.324402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675173c-a30a-4eba-a64e-5d1919ad84d0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.337485 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705529ac-98ec-4230-9e18-aac67f0bfdc1 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.364867 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89f4d4-cb54-442a-b558-87a12d7bd642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.365836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89f4d4-cb54-442a-b558-87a12d7bd642 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.366192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89f4d4-cb54-442a-b558-87a12d7bd642 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.366531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89f4d4-cb54-442a-b558-87a12d7bd642 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.367271 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89f4d4-cb54-442a-b558-87a12d7bd642 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:09.367636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89f4d4-cb54-442a-b558-87a12d7bd642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:09.369082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89f4d4-cb54-442a-b558-87a12d7bd642 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:09.380120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 179/355] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:09.381090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89f4d4-cb54-442a-b558-87a12d7bd642 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.382824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb910cdc-d327-4f95-9221-dfb0323ff700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.383677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb910cdc-d327-4f95-9221-dfb0323ff700 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.384021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb910cdc-d327-4f95-9221-dfb0323ff700 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.384362 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb910cdc-d327-4f95-9221-dfb0323ff700 None admin] PATH_INFO: `/v3/users/ab021469c4364a4ebbfa032cf55b5b64` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.385722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb910cdc-d327-4f95-9221-dfb0323ff700 None admin] RBAC: Authorizing `identity:get_user(user_id=ab021469c4364a4ebbfa032cf55b5b64)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:09.387254 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb910cdc-d327-4f95-9221-dfb0323ff700 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:09.388949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 177/356] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/users/ab021469c4364a4ebbfa032cf55b5b64 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:09.389707 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb910cdc-d327-4f95-9221-dfb0323ff700 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.423689 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.502517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89f4d4-cb54-442a-b558-87a12d7bd642 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.515011 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb910cdc-d327-4f95-9221-dfb0323ff700 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.534203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.541306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230a074-5074-40d8-bd25-d6695ad874b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.542055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230a074-5074-40d8-bd25-d6695ad874b7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.542387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230a074-5074-40d8-bd25-d6695ad874b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.542667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230a074-5074-40d8-bd25-d6695ad874b7 None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.543801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230a074-5074-40d8-bd25-d6695ad874b7 None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:09.549280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230a074-5074-40d8-bd25-d6695ad874b7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:09.551197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 180/357] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:09.552805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde31754-4c90-40f6-bd8b-6005788e3758 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.553317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde31754-4c90-40f6-bd8b-6005788e3758 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.553860 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde31754-4c90-40f6-bd8b-6005788e3758 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.555211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 181/358] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:09.555987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde31754-4c90-40f6-bd8b-6005788e3758 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.566941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98202d09-c161-461e-83c9-e3129874697a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.567610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98202d09-c161-461e-83c9-e3129874697a None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.567888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98202d09-c161-461e-83c9-e3129874697a None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.568159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98202d09-c161-461e-83c9-e3129874697a None admin] PATH_INFO: `/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.577004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98202d09-c161-461e-83c9-e3129874697a None admin] RBAC: Authorizing `identity:get_group(group_id=53dc0b2b17f14665824a5ddbd7e677ca)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:09.579290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98202d09-c161-461e-83c9-e3129874697a None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:09.582001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 178/359] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:09.582709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98202d09-c161-461e-83c9-e3129874697a None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.673410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde31754-4c90-40f6-bd8b-6005788e3758 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.675200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.700496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98202d09-c161-461e-83c9-e3129874697a None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.721513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bccc19-9240-4daf-b2aa-7ebca172ec8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.722167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bccc19-9240-4daf-b2aa-7ebca172ec8a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.722449 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bccc19-9240-4daf-b2aa-7ebca172ec8a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.722715 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bccc19-9240-4daf-b2aa-7ebca172ec8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.748568 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f886ac5-f6b3-4da2-a456-f027ee785a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.749173 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f886ac5-f6b3-4da2-a456-f027ee785a85 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.749448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f886ac5-f6b3-4da2-a456-f027ee785a85 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.749714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f886ac5-f6b3-4da2-a456-f027ee785a85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.750548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f886ac5-f6b3-4da2-a456-f027ee785a85 None admin] RBAC: Adding query filter params (role.id=eeefde0ca97549ed9cd98300215a29ad, user.id=ab021469c4364a4ebbfa032cf55b5b64, scope.system=all) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:09.750853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f886ac5-f6b3-4da2-a456-f027ee785a85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:09.752626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f886ac5-f6b3-4da2-a456-f027ee785a85 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:09.766765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 179/360] 10.222.0.55 () {56 vars in 1308 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/role_assignments?user.id=ab021469c4364a4ebbfa032cf55b5b64&scope.system=all&role.id=eeefde0ca97549ed9cd98300215a29ad => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:09.767611 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f886ac5-f6b3-4da2-a456-f027ee785a85 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.824566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bccc19-9240-4daf-b2aa-7ebca172ec8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:09.826230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bccc19-9240-4daf-b2aa-7ebca172ec8a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:09.828631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.908952 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f886ac5-f6b3-4da2-a456-f027ee785a85 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.960780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd815ad-f0dd-4d10-9f4e-e8e8533955df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.961431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd815ad-f0dd-4d10-9f4e-e8e8533955df None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.961716 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd815ad-f0dd-4d10-9f4e-e8e8533955df None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.962735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd815ad-f0dd-4d10-9f4e-e8e8533955df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.974363 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 182/361] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:09.975230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bccc19-9240-4daf-b2aa-7ebca172ec8a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.978720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.982725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd815ad-f0dd-4d10-9f4e-e8e8533955df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-2fd815ad-f0dd-4d10-9f4e-e8e8533955df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:09.987799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:09.988777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:09.990061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:09.990061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:09.990061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:09.990061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:09.990061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:09.990061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:09.990061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:09.990061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:09.990061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:09.990061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:09.990061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:09.990061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:09.990061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 180/362] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:09.990793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916250f7-f049-4d60-b93d-ce3cca493e9d None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.990793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916250f7-f049-4d60-b93d-ce3cca493e9d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.990793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916250f7-f049-4d60-b93d-ce3cca493e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.012489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-916250f7-f049-4d60-b93d-ce3cca493e9d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:10.013622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-916250f7-f049-4d60-b93d-ce3cca493e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:10.100289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 181/363] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:10.101609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916250f7-f049-4d60-b93d-ce3cca493e9d None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.108551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bccc19-9240-4daf-b2aa-7ebca172ec8a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.133241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.153482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d6a39-74eb-4a87-be1b-85c832b63eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.154857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d6a39-74eb-4a87-be1b-85c832b63eff None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.155210 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d6a39-74eb-4a87-be1b-85c832b63eff None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.155556 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d6a39-74eb-4a87-be1b-85c832b63eff None admin] PATH_INFO: `/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.156770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d6a39-74eb-4a87-be1b-85c832b63eff None admin] RBAC: Authorizing `identity:get_group(group_id=53dc0b2b17f14665824a5ddbd7e677ca)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.158465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d6a39-74eb-4a87-be1b-85c832b63eff None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.160434 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 183/364] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.161080 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d6a39-74eb-4a87-be1b-85c832b63eff None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.216659 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916250f7-f049-4d60-b93d-ce3cca493e9d None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.248716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.262691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b721d-cb83-45ae-b69c-535043fcc0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.264091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b721d-cb83-45ae-b69c-535043fcc0d1 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.264521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b721d-cb83-45ae-b69c-535043fcc0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.264954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b721d-cb83-45ae-b69c-535043fcc0d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.265869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b721d-cb83-45ae-b69c-535043fcc0d1 None admin] RBAC: Adding query filter params (group.id=53dc0b2b17f14665824a5ddbd7e677ca, role.id=8abe8932c5974e7db6fb974c588a9d70, scope.project.id=32e961e117e24576acf2592d4ee51e41) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:10.266347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b721d-cb83-45ae-b69c-535043fcc0d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.268012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b721d-cb83-45ae-b69c-535043fcc0d1 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.277289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 182/365] 10.222.0.55 () {56 vars in 1376 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/role_assignments?group.id=53dc0b2b17f14665824a5ddbd7e677ca&scope.project.id=32e961e117e24576acf2592d4ee51e41&role.id=8abe8932c5974e7db6fb974c588a9d70 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.277289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b721d-cb83-45ae-b69c-535043fcc0d1 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.286235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d6a39-74eb-4a87-be1b-85c832b63eff None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.332397 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f4bc2-33c8-4be5-9de4-43d642043022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.333055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f4bc2-33c8-4be5-9de4-43d642043022 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.333376 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f4bc2-33c8-4be5-9de4-43d642043022 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.333611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f4bc2-33c8-4be5-9de4-43d642043022 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.334885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f4bc2-33c8-4be5-9de4-43d642043022 None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.336519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f4bc2-33c8-4be5-9de4-43d642043022 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.338441 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 184/366] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.339100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f4bc2-33c8-4be5-9de4-43d642043022 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.405121 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.421374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b721d-cb83-45ae-b69c-535043fcc0d1 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.473213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305e4f8-d738-47f8-ab80-e43ec95cbca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.474058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305e4f8-d738-47f8-ab80-e43ec95cbca0 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.474341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305e4f8-d738-47f8-ab80-e43ec95cbca0 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.474612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305e4f8-d738-47f8-ab80-e43ec95cbca0 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.475292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305e4f8-d738-47f8-ab80-e43ec95cbca0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:10.475593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305e4f8-d738-47f8-ab80-e43ec95cbca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.476967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305e4f8-d738-47f8-ab80-e43ec95cbca0 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.481380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f4bc2-33c8-4be5-9de4-43d642043022 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.487993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 183/367] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.488641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305e4f8-d738-47f8-ab80-e43ec95cbca0 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.526001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe240a2b-8aae-47e8-8c84-213303753194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.526549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe240a2b-8aae-47e8-8c84-213303753194 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.526821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe240a2b-8aae-47e8-8c84-213303753194 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.527089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe240a2b-8aae-47e8-8c84-213303753194 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.527819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe240a2b-8aae-47e8-8c84-213303753194 None admin] RBAC: Adding query filter params (group.id=53dc0b2b17f14665824a5ddbd7e677ca, role.id=8abe8932c5974e7db6fb974c588a9d70, scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:10.528125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe240a2b-8aae-47e8-8c84-213303753194 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.529580 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe240a2b-8aae-47e8-8c84-213303753194 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.536691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 185/368] 10.222.0.55 () {56 vars in 1376 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/role_assignments?group.id=53dc0b2b17f14665824a5ddbd7e677ca&scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd&role.id=8abe8932c5974e7db6fb974c588a9d70 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.537097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe240a2b-8aae-47e8-8c84-213303753194 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.547714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.624660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305e4f8-d738-47f8-ab80-e43ec95cbca0 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.671974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c2925-f0aa-4767-ae42-74d80cabe162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.673103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c2925-f0aa-4767-ae42-74d80cabe162 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.673391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c2925-f0aa-4767-ae42-74d80cabe162 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.673667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c2925-f0aa-4767-ae42-74d80cabe162 None admin] PATH_INFO: `/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.674908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c2925-f0aa-4767-ae42-74d80cabe162 None admin] RBAC: Authorizing `identity:get_group(group_id=53dc0b2b17f14665824a5ddbd7e677ca)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.676390 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c2925-f0aa-4767-ae42-74d80cabe162 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.676931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe240a2b-8aae-47e8-8c84-213303753194 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.678411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 184/369] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.679606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c2925-f0aa-4767-ae42-74d80cabe162 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.720441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.727234 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8948b-b796-47c6-aeaa-3c8b4bc108b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.729345 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8948b-b796-47c6-aeaa-3c8b4bc108b8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.729972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8948b-b796-47c6-aeaa-3c8b4bc108b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.730447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8948b-b796-47c6-aeaa-3c8b4bc108b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.748146 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8948b-b796-47c6-aeaa-3c8b4bc108b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-80d8948b-b796-47c6-aeaa-3c8b4bc108b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:10.754798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:10.755824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:10.756641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:10.756641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:10.756641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:10.756641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:10.756641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:10.756641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:10.756641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:10.756641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:10.756641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:10.756641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:10.756641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:10.756641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:10.759989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 186/370] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:10.761283 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8948b-b796-47c6-aeaa-3c8b4bc108b8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.828626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c2925-f0aa-4767-ae42-74d80cabe162 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.867367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442e108-67ae-40f2-b5e8-047ba5ce7ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.868295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442e108-67ae-40f2-b5e8-047ba5ce7ed4 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.868763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442e108-67ae-40f2-b5e8-047ba5ce7ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.869205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442e108-67ae-40f2-b5e8-047ba5ce7ed4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.870839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.932671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8948b-b796-47c6-aeaa-3c8b4bc108b8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.968842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442e108-67ae-40f2-b5e8-047ba5ce7ed4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.970632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442e108-67ae-40f2-b5e8-047ba5ce7ed4 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.983697 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9871573-01cb-4870-9234-04896e7edad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.985475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9871573-01cb-4870-9234-04896e7edad7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.986013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9871573-01cb-4870-9234-04896e7edad7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.986511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9871573-01cb-4870-9234-04896e7edad7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.987470 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9871573-01cb-4870-9234-04896e7edad7 None admin] RBAC: Adding query filter params (group.id=53dc0b2b17f14665824a5ddbd7e677ca, role.id=7d0ac07c6114438da47dcfefeb1fe05b, scope.project.id=32e961e117e24576acf2592d4ee51e41) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:10.988381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9871573-01cb-4870-9234-04896e7edad7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.990627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9871573-01cb-4870-9234-04896e7edad7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.999108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 187/371] 10.222.0.55 () {56 vars in 1376 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/role_assignments?group.id=53dc0b2b17f14665824a5ddbd7e677ca&scope.project.id=32e961e117e24576acf2592d4ee51e41&role.id=7d0ac07c6114438da47dcfefeb1fe05b => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:11.000834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9871573-01cb-4870-9234-04896e7edad7 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:11.004205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:11.160292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 185/372] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:11.169118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442e108-67ae-40f2-b5e8-047ba5ce7ed4 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:11.213771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9871573-01cb-4870-9234-04896e7edad7 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.270927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:11.292954 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465c087-b8ed-4156-a55b-65698274e7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:11.293589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465c087-b8ed-4156-a55b-65698274e7e4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.293889 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465c087-b8ed-4156-a55b-65698274e7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.294666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465c087-b8ed-4156-a55b-65698274e7e4 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.295354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465c087-b8ed-4156-a55b-65698274e7e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:11.295657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465c087-b8ed-4156-a55b-65698274e7e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:11.297044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b465c087-b8ed-4156-a55b-65698274e7e4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:11.308446 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442e108-67ae-40f2-b5e8-047ba5ce7ed4 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.317104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 188/373] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:11.323115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465c087-b8ed-4156-a55b-65698274e7e4 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:11.354295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:11.371817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111477e0-47d7-4445-9e58-c8cff559dde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:11.373013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111477e0-47d7-4445-9e58-c8cff559dde5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.373290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111477e0-47d7-4445-9e58-c8cff559dde5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.373561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111477e0-47d7-4445-9e58-c8cff559dde5 None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.374815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111477e0-47d7-4445-9e58-c8cff559dde5 None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:11.376359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111477e0-47d7-4445-9e58-c8cff559dde5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:11.378499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 186/374] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:11 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:11.387019 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111477e0-47d7-4445-9e58-c8cff559dde5 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:11.526585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465c087-b8ed-4156-a55b-65698274e7e4 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.543406 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111477e0-47d7-4445-9e58-c8cff559dde5 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.590734 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afedd8-2c09-4f52-ae7e-d6bb9f777e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:11.591452 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afedd8-2c09-4f52-ae7e-d6bb9f777e62 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.591727 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afedd8-2c09-4f52-ae7e-d6bb9f777e62 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.591992 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92afedd8-2c09-4f52-ae7e-d6bb9f777e62 None admin] PATH_INFO: `/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.593413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92afedd8-2c09-4f52-ae7e-d6bb9f777e62 None admin] RBAC: Authorizing `identity:get_group(group_id=53dc0b2b17f14665824a5ddbd7e677ca)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:11.594944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92afedd8-2c09-4f52-ae7e-d6bb9f777e62 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:11.596625 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 187/375] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:11 2022] GET /identity/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:11.606690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afedd8-2c09-4f52-ae7e-d6bb9f777e62 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:11.620106 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08f700-8e87-421a-b9a8-fc443cc602bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:11.620727 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f700-8e87-421a-b9a8-fc443cc602bf None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.621057 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f700-8e87-421a-b9a8-fc443cc602bf None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.621326 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f700-8e87-421a-b9a8-fc443cc602bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.722916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:11.735640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:11.754260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08f700-8e87-421a-b9a8-fc443cc602bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:11.755716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08f700-8e87-421a-b9a8-fc443cc602bf None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:11.800589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92afedd8-2c09-4f52-ae7e-d6bb9f777e62 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.890068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf5d4b-aef3-4489-b99d-cd3bb8db540a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:11.890983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf5d4b-aef3-4489-b99d-cd3bb8db540a None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.891435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf5d4b-aef3-4489-b99d-cd3bb8db540a None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.891884 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf5d4b-aef3-4489-b99d-cd3bb8db540a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.893012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf5d4b-aef3-4489-b99d-cd3bb8db540a None admin] RBAC: Adding query filter params (group.id=53dc0b2b17f14665824a5ddbd7e677ca, role.id=7d0ac07c6114438da47dcfefeb1fe05b, scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:11.893508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf5d4b-aef3-4489-b99d-cd3bb8db540a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:11.900532 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf5d4b-aef3-4489-b99d-cd3bb8db540a None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:11.908552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 188/376] 10.222.0.55 () {56 vars in 1376 bytes} [Thu Sep 8 13:39:11 2022] GET /identity/v3/role_assignments?group.id=53dc0b2b17f14665824a5ddbd7e677ca&scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd&role.id=7d0ac07c6114438da47dcfefeb1fe05b => generated 260 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:11.935780 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 189/377] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:39:11.947448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf5d4b-aef3-4489-b99d-cd3bb8db540a None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:12.126673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf5d4b-aef3-4489-b99d-cd3bb8db540a None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:12.161691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:12.179806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb006e2c-3162-42f7-b509-065f87ed6697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:12.180450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb006e2c-3162-42f7-b509-065f87ed6697 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:12.180730 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb006e2c-3162-42f7-b509-065f87ed6697 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:12.181003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb006e2c-3162-42f7-b509-065f87ed6697 None admin] PATH_INFO: `/v3/projects/ed368272c55a45f5a7959b08d8add92c` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:12.182160 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb006e2c-3162-42f7-b509-065f87ed6697 None admin] RBAC: Authorizing `identity:get_project(project_id=ed368272c55a45f5a7959b08d8add92c)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:12.183602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb006e2c-3162-42f7-b509-065f87ed6697 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:12.185634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 189/378] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:11 2022] GET /identity/v3/projects/ed368272c55a45f5a7959b08d8add92c => generated 344 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:12.190901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08f700-8e87-421a-b9a8-fc443cc602bf None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:12.297713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:12.308071 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08f700-8e87-421a-b9a8-fc443cc602bf None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:12.353755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdefe34-f9e7-4d55-af60-c941b17b8ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:12.354392 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdefe34-f9e7-4d55-af60-c941b17b8ada None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:12.354665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdefe34-f9e7-4d55-af60-c941b17b8ada None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:12.354930 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdefe34-f9e7-4d55-af60-c941b17b8ada None admin] PATH_INFO: `/v3/groups/bb5d3d933c5c4cdf8129649ee7e6bfcb` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:12.361604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdefe34-f9e7-4d55-af60-c941b17b8ada None admin] RBAC: Authorizing `identity:get_group(group_id=bb5d3d933c5c4cdf8129649ee7e6bfcb)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:12.363071 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdefe34-f9e7-4d55-af60-c941b17b8ada None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:12.369814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 190/379] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:12 2022] GET /identity/v3/groups/bb5d3d933c5c4cdf8129649ee7e6bfcb => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:12.370793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb006e2c-3162-42f7-b509-065f87ed6697 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:12.535123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb006e2c-3162-42f7-b509-065f87ed6697 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:12.579827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5554bb-cc33-45f0-b967-25cfd7216446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:12.580943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5554bb-cc33-45f0-b967-25cfd7216446 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:12.586158 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5554bb-cc33-45f0-b967-25cfd7216446 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:12.586639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5554bb-cc33-45f0-b967-25cfd7216446 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:12.587597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5554bb-cc33-45f0-b967-25cfd7216446 None admin] RBAC: Adding query filter params (group.id=bb5d3d933c5c4cdf8129649ee7e6bfcb, role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=ed368272c55a45f5a7959b08d8add92c) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:12.588105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5554bb-cc33-45f0-b967-25cfd7216446 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:12.589724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5554bb-cc33-45f0-b967-25cfd7216446 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:12.596166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 190/380] 10.222.0.55 () {56 vars in 1376 bytes} [Thu Sep 8 13:39:12 2022] GET /identity/v3/role_assignments?group.id=bb5d3d933c5c4cdf8129649ee7e6bfcb&scope.project.id=ed368272c55a45f5a7959b08d8add92c&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:13.016673 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc64f99-12f6-4d0f-8cfd-bce121c72224 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.017036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc64f99-12f6-4d0f-8cfd-bce121c72224 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.017233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc64f99-12f6-4d0f-8cfd-bce121c72224 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.018984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 191/381] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:13.024218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b192687-32dd-4933-825a-c70f6218d4aa None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.024509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b192687-32dd-4933-825a-c70f6218d4aa None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.024787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b192687-32dd-4933-825a-c70f6218d4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.052840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4b192687-32dd-4933-825a-c70f6218d4aa None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:13.054069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4b192687-32dd-4933-825a-c70f6218d4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:13.126423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:13.147017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 191/382] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:13.376392 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5ab00-a305-48df-a66a-ea3c3aee7123 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.377384 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5ab00-a305-48df-a66a-ea3c3aee7123 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.377905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5ab00-a305-48df-a66a-ea3c3aee7123 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.379645 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 192/383] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:13.385136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a918f-9fe7-410e-8714-7cb7f5d4c0d2 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.385674 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a918f-9fe7-410e-8714-7cb7f5d4c0d2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.386169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a918f-9fe7-410e-8714-7cb7f5d4c0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.409983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-907a918f-9fe7-410e-8714-7cb7f5d4c0d2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:13.411513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-907a918f-9fe7-410e-8714-7cb7f5d4c0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:13.450088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5ab00-a305-48df-a66a-ea3c3aee7123 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.487675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 192/384] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:13.589379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5ab00-a305-48df-a66a-ea3c3aee7123 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:13.624040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51501756-c9e0-4616-ad6e-ed7041e58588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:13.624681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51501756-c9e0-4616-ad6e-ed7041e58588 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.624954 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51501756-c9e0-4616-ad6e-ed7041e58588 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.625227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51501756-c9e0-4616-ad6e-ed7041e58588 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.636228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51501756-c9e0-4616-ad6e-ed7041e58588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-51501756-c9e0-4616-ad6e-ed7041e58588 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:13.641107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:13.642111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:13.642947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:13.642947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:13.642947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:13.642947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:13.642947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:13.642947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:13.642947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:13.642947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:13.642947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:13.642947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:13.642947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:13.642947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:13.642947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 193/385] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:13.651111 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a918f-9fe7-410e-8714-7cb7f5d4c0d2 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.808633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a918f-9fe7-410e-8714-7cb7f5d4c0d2 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:13.831221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:13.845953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8ddb7-53e8-42fc-bac1-d5d31ce1ccee None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.846371 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8ddb7-53e8-42fc-bac1-d5d31ce1ccee None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.846736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8ddb7-53e8-42fc-bac1-d5d31ce1ccee None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.848600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 194/386] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:13.853479 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabea711-a80d-472b-9740-89de20614d05 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.853906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabea711-a80d-472b-9740-89de20614d05 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.854288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabea711-a80d-472b-9740-89de20614d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.856977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00bc9f-051a-4460-85cb-17090d6b1bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:13.857598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00bc9f-051a-4460-85cb-17090d6b1bf1 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.857916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00bc9f-051a-4460-85cb-17090d6b1bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.858187 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00bc9f-051a-4460-85cb-17090d6b1bf1 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.858859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00bc9f-051a-4460-85cb-17090d6b1bf1 None admin] RBAC: Adding query filter params (name=member) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:13.859159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00bc9f-051a-4460-85cb-17090d6b1bf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:13.860525 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00bc9f-051a-4460-85cb-17090d6b1bf1 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:13.869709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 193/387] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:13.875715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00bc9f-051a-4460-85cb-17090d6b1bf1 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.890196 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-dabea711-a80d-472b-9740-89de20614d05 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:13.891358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dabea711-a80d-472b-9740-89de20614d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:13.909906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:13.991009 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 195/388] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:13.992460 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabea711-a80d-472b-9740-89de20614d05 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.001435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00bc9f-051a-4460-85cb-17090d6b1bf1 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.047770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04b76c-3adb-4590-ae1c-e5c598e8961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.048410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04b76c-3adb-4590-ae1c-e5c598e8961d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.048745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04b76c-3adb-4590-ae1c-e5c598e8961d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.049023 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04b76c-3adb-4590-ae1c-e5c598e8961d None admin] PATH_INFO: `/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.053468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04b76c-3adb-4590-ae1c-e5c598e8961d None admin] RBAC: Authorizing `identity:get_group(group_id=53dc0b2b17f14665824a5ddbd7e677ca)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.055003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04b76c-3adb-4590-ae1c-e5c598e8961d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.056912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 194/389] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:14.071513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04b76c-3adb-4590-ae1c-e5c598e8961d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.110820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabea711-a80d-472b-9740-89de20614d05 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.148304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9896ea1-394a-4169-a625-906b29b87335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.149346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9896ea1-394a-4169-a625-906b29b87335 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.149869 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9896ea1-394a-4169-a625-906b29b87335 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.150364 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9896ea1-394a-4169-a625-906b29b87335 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.166540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.182704 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9896ea1-394a-4169-a625-906b29b87335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-d9896ea1-394a-4169-a625-906b29b87335 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:14.195509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:14.196422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:14.197260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:14.197260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:14.197260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:14.197260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:14.197260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:14.197260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:14.197260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:14.197260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:14.197260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:14.197260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:14.197260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:14.197260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:14.202458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 196/390] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:14.207595 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9896ea1-394a-4169-a625-906b29b87335 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.216988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.268908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04b76c-3adb-4590-ae1c-e5c598e8961d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.323512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c1060-b5b9-4c0f-9f81-95f452010141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.324166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c1060-b5b9-4c0f-9f81-95f452010141 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.324450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c1060-b5b9-4c0f-9f81-95f452010141 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.324724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c1060-b5b9-4c0f-9f81-95f452010141 None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.326432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c1060-b5b9-4c0f-9f81-95f452010141 None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.327967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c1060-b5b9-4c0f-9f81-95f452010141 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.329977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 195/391] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:14.336695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c1060-b5b9-4c0f-9f81-95f452010141 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.359265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9896ea1-394a-4169-a625-906b29b87335 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.405375 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fb1fe-c569-4a11-85b0-9e44f13ea303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.407010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fb1fe-c569-4a11-85b0-9e44f13ea303 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.407589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fb1fe-c569-4a11-85b0-9e44f13ea303 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.408123 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fb1fe-c569-4a11-85b0-9e44f13ea303 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.409203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fb1fe-c569-4a11-85b0-9e44f13ea303 None admin] RBAC: Adding query filter params (name=member) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:14.409656 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fb1fe-c569-4a11-85b0-9e44f13ea303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.411530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fb1fe-c569-4a11-85b0-9e44f13ea303 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.422500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 197/392] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:14.423631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fb1fe-c569-4a11-85b0-9e44f13ea303 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.470887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c1060-b5b9-4c0f-9f81-95f452010141 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.485640 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.498993 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.507349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da285ebb-4432-4a33-bcf3-3e9d0ae02ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.508021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da285ebb-4432-4a33-bcf3-3e9d0ae02ba6 None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.508307 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da285ebb-4432-4a33-bcf3-3e9d0ae02ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.508587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da285ebb-4432-4a33-bcf3-3e9d0ae02ba6 None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd/groups/53dc0b2b17f14665824a5ddbd7e677ca/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.510581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da285ebb-4432-4a33-bcf3-3e9d0ae02ba6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b04feb5fcdf4d38baad71c9a87f28fd, group_id=53dc0b2b17f14665824a5ddbd7e677ca, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.512103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da285ebb-4432-4a33-bcf3-3e9d0ae02ba6 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.529645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 196/393] 10.222.0.55 () {58 vars in 1442 bytes} [Thu Sep 8 13:39:14 2022] PUT /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd/groups/53dc0b2b17f14665824a5ddbd7e677ca/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:39:14.530563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da285ebb-4432-4a33-bcf3-3e9d0ae02ba6 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.561670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fb1fe-c569-4a11-85b0-9e44f13ea303 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.634165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4140403-a065-41db-9dde-4e68a728c43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.634822 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4140403-a065-41db-9dde-4e68a728c43c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.635170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4140403-a065-41db-9dde-4e68a728c43c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.635450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4140403-a065-41db-9dde-4e68a728c43c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.652621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4140403-a065-41db-9dde-4e68a728c43c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:14.660041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da285ebb-4432-4a33-bcf3-3e9d0ae02ba6 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-b4140403-a065-41db-9dde-4e68a728c43c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:14.665432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:14.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:14.667747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:14.667747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:14.667747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:14.667747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:14.667747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:14.667747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:14.667747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:14.667747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:14.667747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:14.667747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:14.667747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:14.667747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:14.667747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 198/394] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:14.673596 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4140403-a065-41db-9dde-4e68a728c43c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.707279 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146135dc-b832-4d7d-8697-6331ea28a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.708067 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146135dc-b832-4d7d-8697-6331ea28a9a0 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.708355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146135dc-b832-4d7d-8697-6331ea28a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.708632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146135dc-b832-4d7d-8697-6331ea28a9a0 None admin] PATH_INFO: `/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.709890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146135dc-b832-4d7d-8697-6331ea28a9a0 None admin] RBAC: Authorizing `identity:get_group(group_id=53dc0b2b17f14665824a5ddbd7e677ca)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.711702 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146135dc-b832-4d7d-8697-6331ea28a9a0 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.718111 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 197/395] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:14.720737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146135dc-b832-4d7d-8697-6331ea28a9a0 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.775320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.851779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146135dc-b832-4d7d-8697-6331ea28a9a0 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.870804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4140403-a065-41db-9dde-4e68a728c43c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.904907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28077f35-59c6-4565-b35b-68ec52b931c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.905615 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28077f35-59c6-4565-b35b-68ec52b931c4 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.905937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28077f35-59c6-4565-b35b-68ec52b931c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.906267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28077f35-59c6-4565-b35b-68ec52b931c4 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.907118 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.908456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28077f35-59c6-4565-b35b-68ec52b931c4 None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.910709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28077f35-59c6-4565-b35b-68ec52b931c4 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.913949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 198/396] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:14.915979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043224b5-2f41-49b2-be90-12ebcfbca140 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.918131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043224b5-2f41-49b2-be90-12ebcfbca140 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.918418 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043224b5-2f41-49b2-be90-12ebcfbca140 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.924042 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 199/397] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:14.924042 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043224b5-2f41-49b2-be90-12ebcfbca140 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.927038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff986b46-0774-499f-95e4-58ef1521d276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.927665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff986b46-0774-499f-95e4-58ef1521d276 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.927942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff986b46-0774-499f-95e4-58ef1521d276 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.928213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff986b46-0774-499f-95e4-58ef1521d276 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.928884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff986b46-0774-499f-95e4-58ef1521d276 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:14.929186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff986b46-0774-499f-95e4-58ef1521d276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.931294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff986b46-0774-499f-95e4-58ef1521d276 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.942075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 199/398] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:14.947793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e000f6-d609-4448-ab01-3019fdd3cd6e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.947793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e000f6-d609-4448-ab01-3019fdd3cd6e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.947793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e000f6-d609-4448-ab01-3019fdd3cd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.964211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-91e000f6-d609-4448-ab01-3019fdd3cd6e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:14.966665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-91e000f6-d609-4448-ab01-3019fdd3cd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:15.041364 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043224b5-2f41-49b2-be90-12ebcfbca140 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.042663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 200/399] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:15.046759 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e000f6-d609-4448-ab01-3019fdd3cd6e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.056587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:15.086085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c336d5-d8fb-4392-9810-c7c8c8e7fb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.087002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c336d5-d8fb-4392-9810-c7c8c8e7fb9e None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.087459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c336d5-d8fb-4392-9810-c7c8c8e7fb9e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.087865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c336d5-d8fb-4392-9810-c7c8c8e7fb9e None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41/groups/53dc0b2b17f14665824a5ddbd7e677ca/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.093914 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c336d5-d8fb-4392-9810-c7c8c8e7fb9e None admin] RBAC: Authorizing `identity:create_grant(project_id=32e961e117e24576acf2592d4ee51e41, group_id=53dc0b2b17f14665824a5ddbd7e677ca, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.095908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c336d5-d8fb-4392-9810-c7c8c8e7fb9e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.118573 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 200/400] 10.222.0.55 () {58 vars in 1442 bytes} [Thu Sep 8 13:39:14 2022] PUT /identity/v3/projects/32e961e117e24576acf2592d4ee51e41/groups/53dc0b2b17f14665824a5ddbd7e677ca/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:15.208985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e000f6-d609-4448-ab01-3019fdd3cd6e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.227521 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:15.245541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981d4aa-abc0-4d17-96a1-b50dc08a213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.246207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981d4aa-abc0-4d17-96a1-b50dc08a213b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.246482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981d4aa-abc0-4d17-96a1-b50dc08a213b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.246752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981d4aa-abc0-4d17-96a1-b50dc08a213b None admin] PATH_INFO: `/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.247922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981d4aa-abc0-4d17-96a1-b50dc08a213b None admin] RBAC: Authorizing `identity:get_group(group_id=53dc0b2b17f14665824a5ddbd7e677ca)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.249367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981d4aa-abc0-4d17-96a1-b50dc08a213b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.251188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 201/401] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:15.257016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c336d5-d8fb-4392-9810-c7c8c8e7fb9e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.384633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c336d5-d8fb-4392-9810-c7c8c8e7fb9e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.426630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57b827-8a0d-42ae-87d5-f97f54f342b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.427771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57b827-8a0d-42ae-87d5-f97f54f342b0 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.429328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981d4aa-abc0-4d17-96a1-b50dc08a213b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.429838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57b827-8a0d-42ae-87d5-f97f54f342b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.430148 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57b827-8a0d-42ae-87d5-f97f54f342b0 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.433968 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57b827-8a0d-42ae-87d5-f97f54f342b0 None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.436026 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57b827-8a0d-42ae-87d5-f97f54f342b0 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.438253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 201/402] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:15.445330 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57b827-8a0d-42ae-87d5-f97f54f342b0 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.514217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:15.557572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981d4aa-abc0-4d17-96a1-b50dc08a213b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.590743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57b827-8a0d-42ae-87d5-f97f54f342b0 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.604736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41905be9-1928-4b89-a842-588e16d6b6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.605677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41905be9-1928-4b89-a842-588e16d6b6e3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.606380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41905be9-1928-4b89-a842-588e16d6b6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.607403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41905be9-1928-4b89-a842-588e16d6b6e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.621721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41905be9-1928-4b89-a842-588e16d6b6e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-41905be9-1928-4b89-a842-588e16d6b6e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:15.628740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:15.629654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:15.630522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:15.630522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:15.630522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:15.630522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:15.630522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:15.630522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:15.630522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:15.630522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:15.630522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:15.630522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:15.630522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:15.630522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:15.635926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f91ae5-9f6e-40d6-8222-abcc8d37a3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.635926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f91ae5-9f6e-40d6-8222-abcc8d37a3e4 None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.635926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f91ae5-9f6e-40d6-8222-abcc8d37a3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.635926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f91ae5-9f6e-40d6-8222-abcc8d37a3e4 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41/groups/53dc0b2b17f14665824a5ddbd7e677ca/roles/7d0ac07c6114438da47dcfefeb1fe05b` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.636240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 202/403] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:15.636240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935abb2d-2acd-4aa8-a925-0ba0efe7bb2f None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.636240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935abb2d-2acd-4aa8-a925-0ba0efe7bb2f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.636240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935abb2d-2acd-4aa8-a925-0ba0efe7bb2f None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.638172 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f91ae5-9f6e-40d6-8222-abcc8d37a3e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=32e961e117e24576acf2592d4ee51e41, group_id=53dc0b2b17f14665824a5ddbd7e677ca, role_id=7d0ac07c6114438da47dcfefeb1fe05b)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.639619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f91ae5-9f6e-40d6-8222-abcc8d37a3e4 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.645359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 203/404] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:15.646723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935abb2d-2acd-4aa8-a925-0ba0efe7bb2f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.657717 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 202/405] 10.222.0.55 () {58 vars in 1442 bytes} [Thu Sep 8 13:39:15 2022] PUT /identity/v3/projects/32e961e117e24576acf2592d4ee51e41/groups/53dc0b2b17f14665824a5ddbd7e677ca/roles/7d0ac07c6114438da47dcfefeb1fe05b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:39:15.659026 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389069b3-a036-455f-afd5-b2d4c7ccdf86 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.659766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389069b3-a036-455f-afd5-b2d4c7ccdf86 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.659766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389069b3-a036-455f-afd5-b2d4c7ccdf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.680018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-389069b3-a036-455f-afd5-b2d4c7ccdf86 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:15.681226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-389069b3-a036-455f-afd5-b2d4c7ccdf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:15.733003 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:15.798787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 203/406] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:15.835285 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935abb2d-2acd-4aa8-a925-0ba0efe7bb2f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.876443 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fdf7e-81cc-4dc8-9e9b-508641be25ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.877066 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fdf7e-81cc-4dc8-9e9b-508641be25ee None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.877404 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fdf7e-81cc-4dc8-9e9b-508641be25ee None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.877680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058fdf7e-81cc-4dc8-9e9b-508641be25ee None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.878452 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fdf7e-81cc-4dc8-9e9b-508641be25ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:15.878757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fdf7e-81cc-4dc8-9e9b-508641be25ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.880140 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058fdf7e-81cc-4dc8-9e9b-508641be25ee None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.889372 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 204/407] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:15.895076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389069b3-a036-455f-afd5-b2d4c7ccdf86 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.931264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:16.005516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389069b3-a036-455f-afd5-b2d4c7ccdf86 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:16.045827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31846ca-068d-4b2b-bd47-7fa291dcdbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:16.046475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31846ca-068d-4b2b-bd47-7fa291dcdbf3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.046756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31846ca-068d-4b2b-bd47-7fa291dcdbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.047028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31846ca-068d-4b2b-bd47-7fa291dcdbf3 None admin] PATH_INFO: `/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.048172 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31846ca-068d-4b2b-bd47-7fa291dcdbf3 None admin] RBAC: Authorizing `identity:get_group(group_id=53dc0b2b17f14665824a5ddbd7e677ca)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:16.049647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31846ca-068d-4b2b-bd47-7fa291dcdbf3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:16.051200 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 204/408] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:16.058823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fdf7e-81cc-4dc8-9e9b-508641be25ee None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:16.202651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058fdf7e-81cc-4dc8-9e9b-508641be25ee None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:16.224409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:16.241571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31846ca-068d-4b2b-bd47-7fa291dcdbf3 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:16.260747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790edf3-ac2f-4f07-95cd-5804a909ed4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:16.263268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790edf3-ac2f-4f07-95cd-5804a909ed4e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.264005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790edf3-ac2f-4f07-95cd-5804a909ed4e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.265141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790edf3-ac2f-4f07-95cd-5804a909ed4e None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.267876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790edf3-ac2f-4f07-95cd-5804a909ed4e None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:16.269862 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790edf3-ac2f-4f07-95cd-5804a909ed4e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:16.273915 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 205/409] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:16 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:16.279187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790edf3-ac2f-4f07-95cd-5804a909ed4e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:16.401727 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:16.414607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31846ca-068d-4b2b-bd47-7fa291dcdbf3 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:16.461786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790edf3-ac2f-4f07-95cd-5804a909ed4e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:16.476230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371bd79f-f5b6-4e41-b759-997f25bd5a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:16.476870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bd79f-f5b6-4e41-b759-997f25bd5a65 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.477150 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bd79f-f5b6-4e41-b759-997f25bd5a65 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.477423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bd79f-f5b6-4e41-b759-997f25bd5a65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.502311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371bd79f-f5b6-4e41-b759-997f25bd5a65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-371bd79f-f5b6-4e41-b759-997f25bd5a65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:16.512722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:16.513631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:16.514480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:16.514480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:16.514480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:16.514480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:16.514480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:16.514480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:16.514480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:16.514480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:16.514480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:16.514480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:16.514480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:16.514480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:16.515538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 205/410] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:16.520264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371bd79f-f5b6-4e41-b759-997f25bd5a65 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:16.535714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fdd21-2ed0-4f55-a4df-eb03ea0d2ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:16.536487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fdd21-2ed0-4f55-a4df-eb03ea0d2ca6 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.536767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fdd21-2ed0-4f55-a4df-eb03ea0d2ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.537044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fdd21-2ed0-4f55-a4df-eb03ea0d2ca6 None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd/groups/53dc0b2b17f14665824a5ddbd7e677ca/roles/7d0ac07c6114438da47dcfefeb1fe05b` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.540989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fdd21-2ed0-4f55-a4df-eb03ea0d2ca6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9b04feb5fcdf4d38baad71c9a87f28fd, group_id=53dc0b2b17f14665824a5ddbd7e677ca, role_id=7d0ac07c6114438da47dcfefeb1fe05b)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:16.542478 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fdd21-2ed0-4f55-a4df-eb03ea0d2ca6 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:16.557514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 206/411] 10.222.0.55 () {58 vars in 1442 bytes} [Thu Sep 8 13:39:16 2022] PUT /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd/groups/53dc0b2b17f14665824a5ddbd7e677ca/roles/7d0ac07c6114438da47dcfefeb1fe05b => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:16.653966 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371bd79f-f5b6-4e41-b759-997f25bd5a65 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:16.692452 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a839359-03ab-4030-b3a4-11960cb1dc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:16.693072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a839359-03ab-4030-b3a4-11960cb1dc77 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.693351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a839359-03ab-4030-b3a4-11960cb1dc77 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.693618 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a839359-03ab-4030-b3a4-11960cb1dc77 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.694336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a839359-03ab-4030-b3a4-11960cb1dc77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:16.694891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a839359-03ab-4030-b3a4-11960cb1dc77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:16.696290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a839359-03ab-4030-b3a4-11960cb1dc77 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:16.706326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 206/412] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:16.711618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fdd21-2ed0-4f55-a4df-eb03ea0d2ca6 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:16.775303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:16.852110 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fdd21-2ed0-4f55-a4df-eb03ea0d2ca6 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:16.895266 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2eeea0-e753-45b7-8bbf-3e3cce5f2217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:16.895914 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2eeea0-e753-45b7-8bbf-3e3cce5f2217 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.896197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2eeea0-e753-45b7-8bbf-3e3cce5f2217 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.896474 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2eeea0-e753-45b7-8bbf-3e3cce5f2217 None admin] PATH_INFO: `/v3/groups/bb5d3d933c5c4cdf8129649ee7e6bfcb` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.897673 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2eeea0-e753-45b7-8bbf-3e3cce5f2217 None admin] RBAC: Authorizing `identity:get_group(group_id=bb5d3d933c5c4cdf8129649ee7e6bfcb)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:16.899185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2eeea0-e753-45b7-8bbf-3e3cce5f2217 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:16.901072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 207/413] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:16 2022] GET /identity/v3/groups/bb5d3d933c5c4cdf8129649ee7e6bfcb => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:16.907450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a839359-03ab-4030-b3a4-11960cb1dc77 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:16.934842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:17.041117 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a839359-03ab-4030-b3a4-11960cb1dc77 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.087500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23f4a8-0756-4cb6-9f4a-4daaa2095f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.088429 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23f4a8-0756-4cb6-9f4a-4daaa2095f13 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.088888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23f4a8-0756-4cb6-9f4a-4daaa2095f13 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.089369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23f4a8-0756-4cb6-9f4a-4daaa2095f13 None admin] PATH_INFO: `/v3/projects/ed368272c55a45f5a7959b08d8add92c` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.092981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23f4a8-0756-4cb6-9f4a-4daaa2095f13 None admin] RBAC: Authorizing `identity:get_project(project_id=ed368272c55a45f5a7959b08d8add92c)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.094660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23f4a8-0756-4cb6-9f4a-4daaa2095f13 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:17.096934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 207/414] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:16 2022] GET /identity/v3/projects/ed368272c55a45f5a7959b08d8add92c => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:17.102317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2eeea0-e753-45b7-8bbf-3e3cce5f2217 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.214242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2eeea0-e753-45b7-8bbf-3e3cce5f2217 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.217520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:17.252401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063858d1-5f07-4147-a2ee-c068ebfe8a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.253591 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063858d1-5f07-4147-a2ee-c068ebfe8a4c None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.254152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063858d1-5f07-4147-a2ee-c068ebfe8a4c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.254641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063858d1-5f07-4147-a2ee-c068ebfe8a4c None admin] PATH_INFO: `/v3/projects/ed368272c55a45f5a7959b08d8add92c/groups/bb5d3d933c5c4cdf8129649ee7e6bfcb/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.256732 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063858d1-5f07-4147-a2ee-c068ebfe8a4c None admin] RBAC: Authorizing `identity:create_grant(project_id=ed368272c55a45f5a7959b08d8add92c, group_id=bb5d3d933c5c4cdf8129649ee7e6bfcb, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.258361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063858d1-5f07-4147-a2ee-c068ebfe8a4c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:17.275499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 208/415] 10.222.0.55 () {58 vars in 1442 bytes} [Thu Sep 8 13:39:17 2022] PUT /identity/v3/projects/ed368272c55a45f5a7959b08d8add92c/groups/bb5d3d933c5c4cdf8129649ee7e6bfcb/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:17.298917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346c736-603b-483a-bd6d-c617fdc74657 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.299248 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346c736-603b-483a-bd6d-c617fdc74657 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.299475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346c736-603b-483a-bd6d-c617fdc74657 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.301192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 208/416] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:17.307203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652706f-b45d-40d9-b173-e3cbdae322a3 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.307548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652706f-b45d-40d9-b173-e3cbdae322a3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.307829 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652706f-b45d-40d9-b173-e3cbdae322a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.329598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6652706f-b45d-40d9-b173-e3cbdae322a3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:17.330942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6652706f-b45d-40d9-b173-e3cbdae322a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:17.407385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 209/417] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:39:17.687644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346c736-603b-483a-bd6d-c617fdc74657 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.802809 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:17.824740 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346c736-603b-483a-bd6d-c617fdc74657 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.842685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248812b-e79c-4547-8ade-9dddb347038b None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.843183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248812b-e79c-4547-8ade-9dddb347038b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.843258 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248812b-e79c-4547-8ade-9dddb347038b None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.844711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 210/418] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:17.850024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098acff-1102-4094-b995-3933d76c6b0e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.850305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098acff-1102-4094-b995-3933d76c6b0e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.850577 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2098acff-1102-4094-b995-3933d76c6b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.881546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2098acff-1102-4094-b995-3933d76c6b0e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:17.883044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2098acff-1102-4094-b995-3933d76c6b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:17.893215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367c8be-b83d-4424-87a3-3176d3f61628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.897852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367c8be-b83d-4424-87a3-3176d3f61628 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.898328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367c8be-b83d-4424-87a3-3176d3f61628 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.898833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f367c8be-b83d-4424-87a3-3176d3f61628 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.924732 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f367c8be-b83d-4424-87a3-3176d3f61628 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-f367c8be-b83d-4424-87a3-3176d3f61628 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:17.940724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:17.941637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:17.942500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:17.942500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:17.942500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:17.942500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:17.942500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:17.942500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:17.942500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:17.942500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:17.942500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:17.942500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:17.942500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:17.942500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:17.942500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 209/419] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:17.948094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367c8be-b83d-4424-87a3-3176d3f61628 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.057202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f367c8be-b83d-4424-87a3-3176d3f61628 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.093266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb381b-863a-41d8-a424-0eedc32a3d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.093904 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb381b-863a-41d8-a424-0eedc32a3d71 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.094181 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb381b-863a-41d8-a424-0eedc32a3d71 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.094448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb381b-863a-41d8-a424-0eedc32a3d71 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.095465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb381b-863a-41d8-a424-0eedc32a3d71 None admin] RBAC: Adding query filter params (name=member) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:18.095771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb381b-863a-41d8-a424-0eedc32a3d71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.097140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bb381b-863a-41d8-a424-0eedc32a3d71 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.106467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 210/420] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:18.112625 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb381b-863a-41d8-a424-0eedc32a3d71 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.122631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 211/421] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:18.142079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.281216 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32edfb-0dfa-4918-921f-403fe050ff53 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.281411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32edfb-0dfa-4918-921f-403fe050ff53 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.281959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32edfb-0dfa-4918-921f-403fe050ff53 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.286264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 212/422] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:18.297096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac6bfe-d61a-49b5-a76c-ad5c530afc5e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.297667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac6bfe-d61a-49b5-a76c-ad5c530afc5e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.298161 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ac6bfe-d61a-49b5-a76c-ad5c530afc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.310988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb381b-863a-41d8-a424-0eedc32a3d71 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.329324 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-04ac6bfe-d61a-49b5-a76c-ad5c530afc5e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:18.331099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-04ac6bfe-d61a-49b5-a76c-ad5c530afc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:18.366811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.376141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7793de-2f63-4ba0-8718-0c8ca7f22acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.376761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7793de-2f63-4ba0-8718-0c8ca7f22acf None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.377047 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7793de-2f63-4ba0-8718-0c8ca7f22acf None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.377320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7793de-2f63-4ba0-8718-0c8ca7f22acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.452484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7793de-2f63-4ba0-8718-0c8ca7f22acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.454240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7793de-2f63-4ba0-8718-0c8ca7f22acf None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.459706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 213/423] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 08 13:39:18.465102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.479670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac6bfe-d61a-49b5-a76c-ad5c530afc5e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.576011 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 211/424] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:18.583920 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7793de-2f63-4ba0-8718-0c8ca7f22acf None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.678298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ac6bfe-d61a-49b5-a76c-ad5c530afc5e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.741429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b07af0-21a3-4aef-b990-0c3d1acbd34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.742084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b07af0-21a3-4aef-b990-0c3d1acbd34d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.742357 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b07af0-21a3-4aef-b990-0c3d1acbd34d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.742627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b07af0-21a3-4aef-b990-0c3d1acbd34d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.759496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7793de-2f63-4ba0-8718-0c8ca7f22acf None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.768767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b07af0-21a3-4aef-b990-0c3d1acbd34d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:18.771107 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-16b07af0-21a3-4aef-b990-0c3d1acbd34d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:18.774733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:18.775644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:18.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:18.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:18.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:18.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:18.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:18.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:18.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:18.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:18.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:18.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:18.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:39:18.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:18.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 214/425] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:18.777181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b07af0-21a3-4aef-b990-0c3d1acbd34d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.797594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.808962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b73282-1424-4dc3-82f1-2e688a6e2eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.810647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b73282-1424-4dc3-82f1-2e688a6e2eec None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.811067 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b73282-1424-4dc3-82f1-2e688a6e2eec None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.811525 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b73282-1424-4dc3-82f1-2e688a6e2eec None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.812883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b73282-1424-4dc3-82f1-2e688a6e2eec None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.814750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b73282-1424-4dc3-82f1-2e688a6e2eec None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.817491 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 212/426] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:18.818500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b73282-1424-4dc3-82f1-2e688a6e2eec None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.909431 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b07af0-21a3-4aef-b990-0c3d1acbd34d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.940633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b73282-1424-4dc3-82f1-2e688a6e2eec None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.957554 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4c048-1cb3-4990-9aca-ef9f9539863d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.958826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4c048-1cb3-4990-9aca-ef9f9539863d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.959204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4c048-1cb3-4990-9aca-ef9f9539863d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.959570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4c048-1cb3-4990-9aca-ef9f9539863d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.973488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4c048-1cb3-4990-9aca-ef9f9539863d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-42b4c048-1cb3-4990-9aca-ef9f9539863d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:18.979679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:18.980602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:18.982015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:18.982015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:18.982015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:18.982015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:18.982015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:18.982015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:18.982015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:18.982015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:18.982015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:18.982015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:18.982015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:18.982015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:18.982015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 215/427] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:18.982696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4c048-1cb3-4990-9aca-ef9f9539863d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.988562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc9fa6-be90-4cc9-8ad2-3c8f0862d95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.989279 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc9fa6-be90-4cc9-8ad2-3c8f0862d95f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.989553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc9fa6-be90-4cc9-8ad2-3c8f0862d95f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.989876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc9fa6-be90-4cc9-8ad2-3c8f0862d95f None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.990550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc9fa6-be90-4cc9-8ad2-3c8f0862d95f None admin] RBAC: Adding query filter params (name=member) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:18.990847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc9fa6-be90-4cc9-8ad2-3c8f0862d95f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.992720 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc9fa6-be90-4cc9-8ad2-3c8f0862d95f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:19.009172 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 213/428] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:19.009172 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc9fa6-be90-4cc9-8ad2-3c8f0862d95f None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:19.055397 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:19.087908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:19.124687 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4c048-1cb3-4990-9aca-ef9f9539863d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:19.129687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc9fa6-be90-4cc9-8ad2-3c8f0862d95f None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:19.161649 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db91965-2df7-4875-b190-8958746d549b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:19.162022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db91965-2df7-4875-b190-8958746d549b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.162298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db91965-2df7-4875-b190-8958746d549b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.162564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db91965-2df7-4875-b190-8958746d549b None admin] PATH_INFO: `/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.163669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db91965-2df7-4875-b190-8958746d549b None admin] RBAC: Authorizing `identity:get_group(group_id=53dc0b2b17f14665824a5ddbd7e677ca)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:19.165093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db91965-2df7-4875-b190-8958746d549b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:19.166664 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 216/429] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:19.167410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db91965-2df7-4875-b190-8958746d549b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:19.176637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef725cd6-18a8-44ae-9fed-5266fcbeb8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:19.177502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef725cd6-18a8-44ae-9fed-5266fcbeb8e2 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.178017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef725cd6-18a8-44ae-9fed-5266fcbeb8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.178515 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef725cd6-18a8-44ae-9fed-5266fcbeb8e2 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.179367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef725cd6-18a8-44ae-9fed-5266fcbeb8e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:19.179858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef725cd6-18a8-44ae-9fed-5266fcbeb8e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:19.181476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef725cd6-18a8-44ae-9fed-5266fcbeb8e2 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:19.191743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 214/430] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:19.193137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef725cd6-18a8-44ae-9fed-5266fcbeb8e2 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:19.308985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db91965-2df7-4875-b190-8958746d549b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:19.317030 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:19.321084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef725cd6-18a8-44ae-9fed-5266fcbeb8e2 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:19.351328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ca62f-78a3-40e4-bea2-f6a12f29a8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:19.352204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ca62f-78a3-40e4-bea2-f6a12f29a8a5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.352627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ca62f-78a3-40e4-bea2-f6a12f29a8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.353039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ca62f-78a3-40e4-bea2-f6a12f29a8a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.359560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bce0e-2846-41ee-bd89-9a94f4797dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:19.360160 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bce0e-2846-41ee-bd89-9a94f4797dd3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.360437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bce0e-2846-41ee-bd89-9a94f4797dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.360706 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bce0e-2846-41ee-bd89-9a94f4797dd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.361432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bce0e-2846-41ee-bd89-9a94f4797dd3 None admin] RBAC: Adding query filter params (group.id=53dc0b2b17f14665824a5ddbd7e677ca, role.id=8abe8932c5974e7db6fb974c588a9d70, scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:19.361737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bce0e-2846-41ee-bd89-9a94f4797dd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:19.363183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bce0e-2846-41ee-bd89-9a94f4797dd3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:19.367083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:19.377763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 215/431] 10.222.0.55 () {56 vars in 1376 bytes} [Thu Sep 8 13:39:19 2022] GET /identity/v3/role_assignments?group.id=53dc0b2b17f14665824a5ddbd7e677ca&scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd&role.id=8abe8932c5974e7db6fb974c588a9d70 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:19.377763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bce0e-2846-41ee-bd89-9a94f4797dd3 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:19.446260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ca62f-78a3-40e4-bea2-f6a12f29a8a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:19.448415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ca62f-78a3-40e4-bea2-f6a12f29a8a5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:19.535770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bce0e-2846-41ee-bd89-9a94f4797dd3 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:19.565963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 217/432] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:19.566277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8964f2f-274f-4a8f-b644-5129138e62ce None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.566549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8964f2f-274f-4a8f-b644-5129138e62ce None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.566768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8964f2f-274f-4a8f-b644-5129138e62ce None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.567993 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 218/433] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:19.574567 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8964f2f-274f-4a8f-b644-5129138e62ce None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:19.587926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5b046-3272-48c8-a7ee-c928b72b975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:19.588962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5b046-3272-48c8-a7ee-c928b72b975b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.589122 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5b046-3272-48c8-a7ee-c928b72b975b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.589391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5b046-3272-48c8-a7ee-c928b72b975b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.628291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:19.631366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:19.674721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5b046-3272-48c8-a7ee-c928b72b975b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:19.676188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5b046-3272-48c8-a7ee-c928b72b975b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:19.702625 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8964f2f-274f-4a8f-b644-5129138e62ce None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:19.737429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7aad7-d183-46bb-ad73-4dce8713fcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:19.738089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7aad7-d183-46bb-ad73-4dce8713fcd2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.738368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7aad7-d183-46bb-ad73-4dce8713fcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.738636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7aad7-d183-46bb-ad73-4dce8713fcd2 None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.739786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7aad7-d183-46bb-ad73-4dce8713fcd2 None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:19.741203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7aad7-d183-46bb-ad73-4dce8713fcd2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:19.747632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 219/434] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:19 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:19.747632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bc51c-32ae-4b9c-a5ab-858b8d6d022c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.747632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bc51c-32ae-4b9c-a5ab-858b8d6d022c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.747632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114bc51c-32ae-4b9c-a5ab-858b8d6d022c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.764024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-114bc51c-32ae-4b9c-a5ab-858b8d6d022c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:19.765207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-114bc51c-32ae-4b9c-a5ab-858b8d6d022c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:19.785640 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 216/435] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:19.787033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5b046-3272-48c8-a7ee-c928b72b975b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:19.839582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 220/436] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:19.840783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bc51c-32ae-4b9c-a5ab-858b8d6d022c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:19.870147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:19.901907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:19.928030 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5b046-3272-48c8-a7ee-c928b72b975b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:19.962889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38729f75-1637-4825-8475-b6fed1eb1269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:19.963439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38729f75-1637-4825-8475-b6fed1eb1269 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.963714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38729f75-1637-4825-8475-b6fed1eb1269 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.963982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38729f75-1637-4825-8475-b6fed1eb1269 None admin] PATH_INFO: `/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.965070 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38729f75-1637-4825-8475-b6fed1eb1269 None admin] RBAC: Authorizing `identity:get_group(group_id=53dc0b2b17f14665824a5ddbd7e677ca)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:19.966553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38729f75-1637-4825-8475-b6fed1eb1269 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:19.970055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 217/437] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:19 2022] GET /identity/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:19.970055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644f66c-4714-4904-bbad-699e15fd36cd None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.970055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644f66c-4714-4904-bbad-699e15fd36cd None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.970055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644f66c-4714-4904-bbad-699e15fd36cd None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.971375 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 218/438] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:19.973255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114bc51c-32ae-4b9c-a5ab-858b8d6d022c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:19.973702 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644f66c-4714-4904-bbad-699e15fd36cd None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.011912 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa64c0-b2e6-4b12-ba75-a691d869c40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.013264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa64c0-b2e6-4b12-ba75-a691d869c40b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.013709 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa64c0-b2e6-4b12-ba75-a691d869c40b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.014155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa64c0-b2e6-4b12-ba75-a691d869c40b None admin] PATH_INFO: `/v3/projects/32e961e117e24576acf2592d4ee51e41` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.015447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa64c0-b2e6-4b12-ba75-a691d869c40b None admin] RBAC: Authorizing `identity:get_project(project_id=32e961e117e24576acf2592d4ee51e41)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:20.017150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa64c0-b2e6-4b12-ba75-a691d869c40b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:20.019114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 221/439] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:19 2022] GET /identity/v3/projects/32e961e117e24576acf2592d4ee51e41 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:20.020340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57449ba3-0e8d-4003-87aa-ff13673ed026 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.020617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57449ba3-0e8d-4003-87aa-ff13673ed026 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.020884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57449ba3-0e8d-4003-87aa-ff13673ed026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.040532 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-57449ba3-0e8d-4003-87aa-ff13673ed026 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:20.041690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-57449ba3-0e8d-4003-87aa-ff13673ed026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:20.084841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644f66c-4714-4904-bbad-699e15fd36cd None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:20.117893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 222/440] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:20.117893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57449ba3-0e8d-4003-87aa-ff13673ed026 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.121234 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:20.127585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f884eb-a951-444c-b9ed-1f5f6f0b62b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.128547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f884eb-a951-444c-b9ed-1f5f6f0b62b3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.128830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f884eb-a951-444c-b9ed-1f5f6f0b62b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.129101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f884eb-a951-444c-b9ed-1f5f6f0b62b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.130165 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f884eb-a951-444c-b9ed-1f5f6f0b62b3 None admin] RBAC: Adding query filter params (group.id=53dc0b2b17f14665824a5ddbd7e677ca, role.id=8abe8932c5974e7db6fb974c588a9d70, scope.project.id=32e961e117e24576acf2592d4ee51e41) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:20.130749 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f884eb-a951-444c-b9ed-1f5f6f0b62b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:20.132176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f884eb-a951-444c-b9ed-1f5f6f0b62b3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:20.138633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 219/441] 10.222.0.55 () {56 vars in 1376 bytes} [Thu Sep 8 13:39:19 2022] GET /identity/v3/role_assignments?group.id=53dc0b2b17f14665824a5ddbd7e677ca&scope.project.id=32e961e117e24576acf2592d4ee51e41&role.id=8abe8932c5974e7db6fb974c588a9d70 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:20.139261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f884eb-a951-444c-b9ed-1f5f6f0b62b3 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.166278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:20.256513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f884eb-a951-444c-b9ed-1f5f6f0b62b3 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:20.267106 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57449ba3-0e8d-4003-87aa-ff13673ed026 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:20.294993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0460bd-3c4c-47b3-96db-34c96820775f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.295644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0460bd-3c4c-47b3-96db-34c96820775f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.295918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0460bd-3c4c-47b3-96db-34c96820775f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.296186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0460bd-3c4c-47b3-96db-34c96820775f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.308081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0460bd-3c4c-47b3-96db-34c96820775f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:20.314019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59501c6-4e8a-48da-bd7e-6440db9a0b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-6b0460bd-3c4c-47b3-96db-34c96820775f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:20.314543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:20.315436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:20.316258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:20.316258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:20.316258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:20.316258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:20.316258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:20.316258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:20.316258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:20.316258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:20.316258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:20.316258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:20.316258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:20.316258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:20.316914 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59501c6-4e8a-48da-bd7e-6440db9a0b92 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.316914 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59501c6-4e8a-48da-bd7e-6440db9a0b92 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.316914 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59501c6-4e8a-48da-bd7e-6440db9a0b92 None admin] PATH_INFO: `/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.320548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59501c6-4e8a-48da-bd7e-6440db9a0b92 None admin] RBAC: Authorizing `identity:get_group(group_id=53dc0b2b17f14665824a5ddbd7e677ca)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:20.320548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59501c6-4e8a-48da-bd7e-6440db9a0b92 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:20.320881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 220/442] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:20.321791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0460bd-3c4c-47b3-96db-34c96820775f None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.322501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 223/443] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:20 2022] GET /identity/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:20.327913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59501c6-4e8a-48da-bd7e-6440db9a0b92 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.385470 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:20.445236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0460bd-3c4c-47b3-96db-34c96820775f None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:20.446415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59501c6-4e8a-48da-bd7e-6440db9a0b92 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:20.455156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:20.484601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730375b2-acdb-4839-8569-1f58b245593e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.485212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730375b2-acdb-4839-8569-1f58b245593e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.485488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730375b2-acdb-4839-8569-1f58b245593e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.485781 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730375b2-acdb-4839-8569-1f58b245593e None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.486133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f0927-1a3f-4bd3-b1ba-15e3baaba32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.486451 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730375b2-acdb-4839-8569-1f58b245593e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:20.486746 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f0927-1a3f-4bd3-b1ba-15e3baaba32e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.486843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730375b2-acdb-4839-8569-1f58b245593e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:20.487879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f0927-1a3f-4bd3-b1ba-15e3baaba32e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.487879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f0927-1a3f-4bd3-b1ba-15e3baaba32e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.488052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f0927-1a3f-4bd3-b1ba-15e3baaba32e None admin] RBAC: Adding query filter params (group.id=53dc0b2b17f14665824a5ddbd7e677ca, role.id=7d0ac07c6114438da47dcfefeb1fe05b, scope.project.id=32e961e117e24576acf2592d4ee51e41) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:20.488128 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730375b2-acdb-4839-8569-1f58b245593e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:20.488345 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f0927-1a3f-4bd3-b1ba-15e3baaba32e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:20.489776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672f0927-1a3f-4bd3-b1ba-15e3baaba32e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:20.496416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 221/445] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:20.497628 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730375b2-acdb-4839-8569-1f58b245593e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.501904 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 224/445] 10.222.0.55 () {56 vars in 1376 bytes} [Thu Sep 8 13:39:20 2022] GET /identity/v3/role_assignments?group.id=53dc0b2b17f14665824a5ddbd7e677ca&scope.project.id=32e961e117e24576acf2592d4ee51e41&role.id=7d0ac07c6114438da47dcfefeb1fe05b => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:20.503641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f0927-1a3f-4bd3-b1ba-15e3baaba32e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.630801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730375b2-acdb-4839-8569-1f58b245593e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:20.643419 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672f0927-1a3f-4bd3-b1ba-15e3baaba32e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:20.655133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:20.682582 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d698c6a-c8a6-492e-bd02-4d7bcb9d5ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.683230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d698c6a-c8a6-492e-bd02-4d7bcb9d5ae0 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.683505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d698c6a-c8a6-492e-bd02-4d7bcb9d5ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.683776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d698c6a-c8a6-492e-bd02-4d7bcb9d5ae0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.690657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4953d-9ffd-4c3e-bc06-2b8109c1b1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.691382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4953d-9ffd-4c3e-bc06-2b8109c1b1b7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.691659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4953d-9ffd-4c3e-bc06-2b8109c1b1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.692033 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4953d-9ffd-4c3e-bc06-2b8109c1b1b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.695410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d698c6a-c8a6-492e-bd02-4d7bcb9d5ae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-6d698c6a-c8a6-492e-bd02-4d7bcb9d5ae0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:20.701647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:20.702616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:20.703682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:20.703682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:20.703682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:20.703682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:20.703682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:20.703682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:20.703682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:20.703682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:20.703682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:20.703682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:20.703682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:20.703682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:20.703682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 222/446] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:20.708465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d698c6a-c8a6-492e-bd02-4d7bcb9d5ae0 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.738324 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:20.774427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4953d-9ffd-4c3e-bc06-2b8109c1b1b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:20.777333 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4953d-9ffd-4c3e-bc06-2b8109c1b1b7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:20.823429 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d698c6a-c8a6-492e-bd02-4d7bcb9d5ae0 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:20.861124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86b282-4edf-47c3-9570-731f62ab1915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.862143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86b282-4edf-47c3-9570-731f62ab1915 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.862143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86b282-4edf-47c3-9570-731f62ab1915 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.862326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86b282-4edf-47c3-9570-731f62ab1915 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.863373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86b282-4edf-47c3-9570-731f62ab1915 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:20.863373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86b282-4edf-47c3-9570-731f62ab1915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:20.864594 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86b282-4edf-47c3-9570-731f62ab1915 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:20.881784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 223/447] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:20.888197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86b282-4edf-47c3-9570-731f62ab1915 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.897374 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 225/448] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:39:20.902696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4953d-9ffd-4c3e-bc06-2b8109c1b1b7 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.945903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:20.989969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.023359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86b282-4edf-47c3-9570-731f62ab1915 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.028132 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4953d-9ffd-4c3e-bc06-2b8109c1b1b7 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.065072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2d45f-9a05-404f-a0fb-b7dba00b155c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.065696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2d45f-9a05-404f-a0fb-b7dba00b155c None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.066012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2d45f-9a05-404f-a0fb-b7dba00b155c None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.066286 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2d45f-9a05-404f-a0fb-b7dba00b155c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.068907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f291d5-d795-4cea-9709-a54ac9f7369c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.069836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f291d5-d795-4cea-9709-a54ac9f7369c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.070119 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f291d5-d795-4cea-9709-a54ac9f7369c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.070385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f291d5-d795-4cea-9709-a54ac9f7369c None admin] PATH_INFO: `/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.071521 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f291d5-d795-4cea-9709-a54ac9f7369c None admin] RBAC: Authorizing `identity:get_project(project_id=9b04feb5fcdf4d38baad71c9a87f28fd)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.073097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f291d5-d795-4cea-9709-a54ac9f7369c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.074809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 226/449] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:20 2022] GET /identity/v3/projects/9b04feb5fcdf4d38baad71c9a87f28fd => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:21.080763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f291d5-d795-4cea-9709-a54ac9f7369c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.135534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2d45f-9a05-404f-a0fb-b7dba00b155c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.136907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2d45f-9a05-404f-a0fb-b7dba00b155c None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.177460 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f291d5-d795-4cea-9709-a54ac9f7369c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.177638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.210806 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90b4cd-ebbc-40ca-b761-101d84d9e67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.211425 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90b4cd-ebbc-40ca-b761-101d84d9e67d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.211702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90b4cd-ebbc-40ca-b761-101d84d9e67d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.211974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90b4cd-ebbc-40ca-b761-101d84d9e67d None admin] PATH_INFO: `/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.213086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90b4cd-ebbc-40ca-b761-101d84d9e67d None admin] RBAC: Authorizing `identity:get_group(group_id=53dc0b2b17f14665824a5ddbd7e677ca)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.214537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90b4cd-ebbc-40ca-b761-101d84d9e67d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.216603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 227/450] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/groups/53dc0b2b17f14665824a5ddbd7e677ca => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:21.222412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90b4cd-ebbc-40ca-b761-101d84d9e67d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.230353 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.236353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 224/451] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:39:21.241801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2d45f-9a05-404f-a0fb-b7dba00b155c None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.321999 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90b4cd-ebbc-40ca-b761-101d84d9e67d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.340829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2d45f-9a05-404f-a0fb-b7dba00b155c None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.355235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fed3f-beb5-481f-a11e-79a202f7895f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.355835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fed3f-beb5-481f-a11e-79a202f7895f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.356134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fed3f-beb5-481f-a11e-79a202f7895f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.356409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fed3f-beb5-481f-a11e-79a202f7895f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.357130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fed3f-beb5-481f-a11e-79a202f7895f None admin] RBAC: Adding query filter params (group.id=53dc0b2b17f14665824a5ddbd7e677ca, role.id=7d0ac07c6114438da47dcfefeb1fe05b, scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:21.357436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fed3f-beb5-481f-a11e-79a202f7895f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.358900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fed3f-beb5-481f-a11e-79a202f7895f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.364533 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 228/452] 10.222.0.55 () {56 vars in 1376 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/role_assignments?group.id=53dc0b2b17f14665824a5ddbd7e677ca&scope.project.id=9b04feb5fcdf4d38baad71c9a87f28fd&role.id=7d0ac07c6114438da47dcfefeb1fe05b => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:21.374839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a60b5-e471-41ee-9055-f579edd924e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.375489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a60b5-e471-41ee-9055-f579edd924e8 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.375768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a60b5-e471-41ee-9055-f579edd924e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.376040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a60b5-e471-41ee-9055-f579edd924e8 None admin] PATH_INFO: `/v3/projects/ed368272c55a45f5a7959b08d8add92c` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.377232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a60b5-e471-41ee-9055-f579edd924e8 None admin] RBAC: Authorizing `identity:get_project(project_id=ed368272c55a45f5a7959b08d8add92c)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.378737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a60b5-e471-41ee-9055-f579edd924e8 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.380620 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 225/453] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/projects/ed368272c55a45f5a7959b08d8add92c => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:21.386352 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fed3f-beb5-481f-a11e-79a202f7895f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.460196 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.493291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fed3f-beb5-481f-a11e-79a202f7895f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.530088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417f7002-bc3f-4bf9-a033-50fad2c7ffe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.530725 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f7002-bc3f-4bf9-a033-50fad2c7ffe1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.531005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f7002-bc3f-4bf9-a033-50fad2c7ffe1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.531281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417f7002-bc3f-4bf9-a033-50fad2c7ffe1 None admin] PATH_INFO: `/v3/groups/bb5d3d933c5c4cdf8129649ee7e6bfcb` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.532409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f7002-bc3f-4bf9-a033-50fad2c7ffe1 None admin] RBAC: Authorizing `identity:get_group(group_id=bb5d3d933c5c4cdf8129649ee7e6bfcb)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.533876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417f7002-bc3f-4bf9-a033-50fad2c7ffe1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.535432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 229/454] 10.222.0.55 () {56 vars in 1180 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/groups/bb5d3d933c5c4cdf8129649ee7e6bfcb => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:21.541371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a60b5-e471-41ee-9055-f579edd924e8 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.552730 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.650141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a60b5-e471-41ee-9055-f579edd924e8 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.684581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edaecb3-7461-45d2-a04f-89d85e2589f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.685185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edaecb3-7461-45d2-a04f-89d85e2589f5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.685465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edaecb3-7461-45d2-a04f-89d85e2589f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.685803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edaecb3-7461-45d2-a04f-89d85e2589f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.686557 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edaecb3-7461-45d2-a04f-89d85e2589f5 None admin] RBAC: Adding query filter params (group.id=bb5d3d933c5c4cdf8129649ee7e6bfcb, role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=ed368272c55a45f5a7959b08d8add92c) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:21.686893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edaecb3-7461-45d2-a04f-89d85e2589f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.688345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edaecb3-7461-45d2-a04f-89d85e2589f5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.694729 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 226/455] 10.222.0.55 () {56 vars in 1376 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/role_assignments?group.id=bb5d3d933c5c4cdf8129649ee7e6bfcb&scope.project.id=ed368272c55a45f5a7959b08d8add92c&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:24.259695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653af77-5f64-4f3a-871c-01971b77101d None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.260666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653af77-5f64-4f3a-871c-01971b77101d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.261167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9653af77-5f64-4f3a-871c-01971b77101d None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.263144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 230/456] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:24.268678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dfd80-3cfb-4d3b-a188-69b5f6a62693 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.269075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dfd80-3cfb-4d3b-a188-69b5f6a62693 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.269217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dfd80-3cfb-4d3b-a188-69b5f6a62693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.295030 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-614dfd80-3cfb-4d3b-a188-69b5f6a62693 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:24.296440 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-614dfd80-3cfb-4d3b-a188-69b5f6a62693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:24.388320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 227/457] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:24.436805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4d1cf-d31e-48cd-aae0-a495ddf96324 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.437086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4d1cf-d31e-48cd-aae0-a495ddf96324 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.437561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4d1cf-d31e-48cd-aae0-a495ddf96324 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.439345 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 231/458] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:24.444236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c269a-7683-411c-82ea-219fea23cfc5 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.444776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c269a-7683-411c-82ea-219fea23cfc5 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.445054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c269a-7683-411c-82ea-219fea23cfc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.451449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:24.466559 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-070c269a-7683-411c-82ea-219fea23cfc5 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:24.467740 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-070c269a-7683-411c-82ea-219fea23cfc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:24.556782 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 228/459] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:24.670582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381a47b-91a9-43d7-bcbf-0284b29a2bc4 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.671245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381a47b-91a9-43d7-bcbf-0284b29a2bc4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.671760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381a47b-91a9-43d7-bcbf-0284b29a2bc4 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.673587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 232/460] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:24.678530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86280aeb-a81b-4eba-9263-12d3bc6a383c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.678820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86280aeb-a81b-4eba-9263-12d3bc6a383c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.679094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86280aeb-a81b-4eba-9263-12d3bc6a383c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.706014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-86280aeb-a81b-4eba-9263-12d3bc6a383c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:24.707202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-86280aeb-a81b-4eba-9263-12d3bc6a383c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:24.724027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381a47b-91a9-43d7-bcbf-0284b29a2bc4 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:24.811090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 229/461] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:24.887688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86280aeb-a81b-4eba-9263-12d3bc6a383c None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:24.893865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381a47b-91a9-43d7-bcbf-0284b29a2bc4 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:24.958238 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686980ce-0fb8-4bff-9a04-79df8a80c93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:24.959268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686980ce-0fb8-4bff-9a04-79df8a80c93a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.959745 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686980ce-0fb8-4bff-9a04-79df8a80c93a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.960237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686980ce-0fb8-4bff-9a04-79df8a80c93a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.977214 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686980ce-0fb8-4bff-9a04-79df8a80c93a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:24.983859 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:24.985324 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686980ce-0fb8-4bff-9a04-79df8a80c93a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.002098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-686980ce-0fb8-4bff-9a04-79df8a80c93a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:25.006997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:25.007927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:25.008808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:25.008808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:25.008808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:25.014545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 233/462] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:25.019276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686980ce-0fb8-4bff-9a04-79df8a80c93a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.073722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86280aeb-a81b-4eba-9263-12d3bc6a383c None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.147907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4333464-26da-494f-a614-dac000fb4b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.148960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4333464-26da-494f-a614-dac000fb4b91 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.149445 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4333464-26da-494f-a614-dac000fb4b91 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.149939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4333464-26da-494f-a614-dac000fb4b91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.173566 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4333464-26da-494f-a614-dac000fb4b91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.175533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4333464-26da-494f-a614-dac000fb4b91 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-e4333464-26da-494f-a614-dac000fb4b91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:25.196696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:25.197655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:25.198856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:25.198856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:25.198856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:25.200937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 230/463] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:25.207691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4333464-26da-494f-a614-dac000fb4b91 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.219832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686980ce-0fb8-4bff-9a04-79df8a80c93a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.265285 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8baa04b-f6f7-4d7d-98ef-07ed99edf69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.266748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8baa04b-f6f7-4d7d-98ef-07ed99edf69d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.267262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8baa04b-f6f7-4d7d-98ef-07ed99edf69d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.267761 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8baa04b-f6f7-4d7d-98ef-07ed99edf69d None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.268621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8baa04b-f6f7-4d7d-98ef-07ed99edf69d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:25.269124 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8baa04b-f6f7-4d7d-98ef-07ed99edf69d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.270880 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8baa04b-f6f7-4d7d-98ef-07ed99edf69d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.284484 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 234/464] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:25.285619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8baa04b-f6f7-4d7d-98ef-07ed99edf69d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.320253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4333464-26da-494f-a614-dac000fb4b91 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.335224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:25.337379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:25.357239 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a3ea3-5547-480a-9a8b-46d31bb48f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.357929 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a3ea3-5547-480a-9a8b-46d31bb48f6b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.358209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a3ea3-5547-480a-9a8b-46d31bb48f6b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.358478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a3ea3-5547-480a-9a8b-46d31bb48f6b None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.359126 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a3ea3-5547-480a-9a8b-46d31bb48f6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:25.359425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a3ea3-5547-480a-9a8b-46d31bb48f6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.360791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a3ea3-5547-480a-9a8b-46d31bb48f6b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.376934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 231/465] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:25.378503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057356f0-df5c-42b7-bb3f-39f8b1d6ffbf None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.378784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057356f0-df5c-42b7-bb3f-39f8b1d6ffbf None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.379057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057356f0-df5c-42b7-bb3f-39f8b1d6ffbf None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.380636 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 232/466] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:25.382444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057356f0-df5c-42b7-bb3f-39f8b1d6ffbf None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.400401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8baa04b-f6f7-4d7d-98ef-07ed99edf69d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.436756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4cba7-cc9c-4232-be25-6bd03541c295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.437475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4cba7-cc9c-4232-be25-6bd03541c295 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.437755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4cba7-cc9c-4232-be25-6bd03541c295 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.438032 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4cba7-cc9c-4232-be25-6bd03541c295 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.448664 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4cba7-cc9c-4232-be25-6bd03541c295 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.450681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4cba7-cc9c-4232-be25-6bd03541c295 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-22a4cba7-cc9c-4232-be25-6bd03541c295 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:25.465716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:25.466717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:25.467570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:25.467570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:25.467570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:25.467570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 235/467] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:25.467570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4cba7-cc9c-4232-be25-6bd03541c295 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.497143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057356f0-df5c-42b7-bb3f-39f8b1d6ffbf None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.532385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc060c8-2ec8-4efb-b75c-13cb063ecccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.532997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc060c8-2ec8-4efb-b75c-13cb063ecccf None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.533360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc060c8-2ec8-4efb-b75c-13cb063ecccf None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.533541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc060c8-2ec8-4efb-b75c-13cb063ecccf None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.534212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc060c8-2ec8-4efb-b75c-13cb063ecccf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.535799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc060c8-2ec8-4efb-b75c-13cb063ecccf None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.559185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 233/468] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:39:25 2022] POST /identity/v3/users => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:25.560723 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0729fae-9d1c-4fd1-95ea-0345ee24e47d None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.561239 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0729fae-9d1c-4fd1-95ea-0345ee24e47d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.561705 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0729fae-9d1c-4fd1-95ea-0345ee24e47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.583914 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-e0729fae-9d1c-4fd1-95ea-0345ee24e47d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:25.585562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e0729fae-9d1c-4fd1-95ea-0345ee24e47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:25.593458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4cba7-cc9c-4232-be25-6bd03541c295 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.598059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:25.615132 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:25.646634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6476014-608b-4801-bc72-aca0607616d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.647258 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6476014-608b-4801-bc72-aca0607616d8 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.647632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6476014-608b-4801-bc72-aca0607616d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.647935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6476014-608b-4801-bc72-aca0607616d8 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.648941 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6476014-608b-4801-bc72-aca0607616d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.651044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6476014-608b-4801-bc72-aca0607616d8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.676857 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 234/469] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:25.678251 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0729fae-9d1c-4fd1-95ea-0345ee24e47d None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.689953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 236/470] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:39:25 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:25.839802 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0729fae-9d1c-4fd1-95ea-0345ee24e47d None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.879779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469681bc-2c96-49b6-b475-967c5f46b1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.880409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469681bc-2c96-49b6-b475-967c5f46b1bb None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.880685 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469681bc-2c96-49b6-b475-967c5f46b1bb None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.880950 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469681bc-2c96-49b6-b475-967c5f46b1bb None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.881603 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469681bc-2c96-49b6-b475-967c5f46b1bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:25.881940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469681bc-2c96-49b6-b475-967c5f46b1bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.883421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469681bc-2c96-49b6-b475-967c5f46b1bb None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.901688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 235/471] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:25.908097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6476014-608b-4801-bc72-aca0607616d8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.951098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da700837-675e-4cb3-a2f8-4ab552c65214 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.951416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da700837-675e-4cb3-a2f8-4ab552c65214 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.951645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da700837-675e-4cb3-a2f8-4ab552c65214 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.953325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 236/472] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:25.962512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d4bc1-3f93-4660-b55b-81d87f9dadf3 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.962798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d4bc1-3f93-4660-b55b-81d87f9dadf3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.963068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d4bc1-3f93-4660-b55b-81d87f9dadf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.982194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-fd5d4bc1-3f93-4660-b55b-81d87f9dadf3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:25.983427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-fd5d4bc1-3f93-4660-b55b-81d87f9dadf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:26.002436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:26.041163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6476014-608b-4801-bc72-aca0607616d8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.077359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 237/473] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:26.078409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d4bc1-3f93-4660-b55b-81d87f9dadf3 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.083924 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4f5c0-3992-4304-a7d7-b91bbeedf1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.084771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4f5c0-3992-4304-a7d7-b91bbeedf1e2 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.085242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4f5c0-3992-4304-a7d7-b91bbeedf1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.085749 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4f5c0-3992-4304-a7d7-b91bbeedf1e2 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.086592 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4f5c0-3992-4304-a7d7-b91bbeedf1e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:26.088222 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4f5c0-3992-4304-a7d7-b91bbeedf1e2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:26.112812 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 237/474] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:39:25 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:26.233220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d4bc1-3f93-4660-b55b-81d87f9dadf3 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.282674 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919d3e8-c824-4f27-a0e1-92df1339f2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.283322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919d3e8-c824-4f27-a0e1-92df1339f2e8 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.283602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919d3e8-c824-4f27-a0e1-92df1339f2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.283870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919d3e8-c824-4f27-a0e1-92df1339f2e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.299493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919d3e8-c824-4f27-a0e1-92df1339f2e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:26.300951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919d3e8-c824-4f27-a0e1-92df1339f2e8 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-0919d3e8-c824-4f27-a0e1-92df1339f2e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:26.320075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:26.321503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:26.322432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:26.322432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:26.322432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:26.322432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 238/475] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:26.330316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4f5c0-3992-4304-a7d7-b91bbeedf1e2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.361948 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:26.390761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919d3e8-c824-4f27-a0e1-92df1339f2e8 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.394326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:26.472063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4f5c0-3992-4304-a7d7-b91bbeedf1e2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.504616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919d3e8-c824-4f27-a0e1-92df1339f2e8 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.507619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f2352-017a-43dd-8b64-8e476f8f6c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.508242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f2352-017a-43dd-8b64-8e476f8f6c20 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.508531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f2352-017a-43dd-8b64-8e476f8f6c20 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.508865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f2352-017a-43dd-8b64-8e476f8f6c20 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.509525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f2352-017a-43dd-8b64-8e476f8f6c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:26.509891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f2352-017a-43dd-8b64-8e476f8f6c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:26.511278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1f2352-017a-43dd-8b64-8e476f8f6c20 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:26.523736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 238/476] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:26.530682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f2352-017a-43dd-8b64-8e476f8f6c20 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.544216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee031c-c495-4805-b62b-abfce491bef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.544834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee031c-c495-4805-b62b-abfce491bef9 None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.545117 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee031c-c495-4805-b62b-abfce491bef9 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.545393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee031c-c495-4805-b62b-abfce491bef9 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.546105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee031c-c495-4805-b62b-abfce491bef9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:26.547491 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee031c-c495-4805-b62b-abfce491bef9 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:26.556409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.sql.core [None req-dfee031c-c495-4805-b62b-abfce491bef9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 13:39:26.556409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 13:39:26.556409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [parameters: {'id': 'c977cb41f9bc497d9718fcaad39f8ab7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 13:39:26.556409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87917) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-dfee031c-c495-4805-b62b-abfce491bef9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application result.read() Sep 08 13:39:26.573907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return ref.to_dict() Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:39:26.574892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application yield resource Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise self.value Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application session.commit() Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 13:39:26.575815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self.session.flush() Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application compat.raise_( Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 13:39:26.576670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:39:26.577538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application result.read() Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:39:26.579890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [parameters: {'id': 'c977cb41f9bc497d9718fcaad39f8ab7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:26.584570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 13:39:26.586605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:26.588499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 239/477] 10.222.0.55 () {60 vars in 1131 bytes} [Thu Sep 8 13:39:26 2022] POST /identity/v3/roles => generated 144 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:26.594835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee031c-c495-4805-b62b-abfce491bef9 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.613136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:26.641180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f2352-017a-43dd-8b64-8e476f8f6c20 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.692011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33871ca-f4e0-4d5c-9e64-1ce948d36d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.695570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33871ca-f4e0-4d5c-9e64-1ce948d36d22 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.695751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:26.695904 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33871ca-f4e0-4d5c-9e64-1ce948d36d22 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.697604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33871ca-f4e0-4d5c-9e64-1ce948d36d22 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.699353 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33871ca-f4e0-4d5c-9e64-1ce948d36d22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:26.700925 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33871ca-f4e0-4d5c-9e64-1ce948d36d22 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:26.740779 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 239/478] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:39:26 2022] POST /identity/v3/users => generated 249 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:26.753952 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee031c-c495-4805-b62b-abfce491bef9 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.791610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f3f3c-7857-4c2d-9481-14dc20849f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.792375 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3f3c-7857-4c2d-9481-14dc20849f5f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.792595 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3f3c-7857-4c2d-9481-14dc20849f5f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.792869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f3f3c-7857-4c2d-9481-14dc20849f5f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.805201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f3f3c-7857-4c2d-9481-14dc20849f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-484f3f3c-7857-4c2d-9481-14dc20849f5f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:26.811175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:26.812052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:26.812869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:26.812869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:26.812869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:26.812869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:26.812869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:26.812869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:26.812869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:26.812869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:26.812869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:26.812869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:26.812869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:39:26.812869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:26.812869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 240/479] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:26.820095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33871ca-f4e0-4d5c-9e64-1ce948d36d22 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.971504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33871ca-f4e0-4d5c-9e64-1ce948d36d22 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:27.011067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:27.020711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793f4eb-5dfc-4be1-9f2b-3e0d2d508c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:27.021347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793f4eb-5dfc-4be1-9f2b-3e0d2d508c82 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.021630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793f4eb-5dfc-4be1-9f2b-3e0d2d508c82 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.021929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793f4eb-5dfc-4be1-9f2b-3e0d2d508c82 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.022611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793f4eb-5dfc-4be1-9f2b-3e0d2d508c82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:27.022914 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793f4eb-5dfc-4be1-9f2b-3e0d2d508c82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:27.024312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793f4eb-5dfc-4be1-9f2b-3e0d2d508c82 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:27.042096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 240/480] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:28.399634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd0891-39a1-4e77-a6c0-9d1ddf5c5a6d None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.400179 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd0891-39a1-4e77-a6c0-9d1ddf5c5a6d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.400179 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd0891-39a1-4e77-a6c0-9d1ddf5c5a6d None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.401817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 241/481] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:28.407722 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94acff-c143-4f6a-85c4-7282c784cbda None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.408190 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94acff-c143-4f6a-85c4-7282c784cbda None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.408288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f94acff-c143-4f6a-85c4-7282c784cbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.428171 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0f94acff-c143-4f6a-85c4-7282c784cbda None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:28.429371 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0f94acff-c143-4f6a-85c4-7282c784cbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:28.529033 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 241/482] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:28.535386 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12bc86-795a-448f-a0f1-3ba4fa5de596 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.535665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12bc86-795a-448f-a0f1-3ba4fa5de596 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.535934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12bc86-795a-448f-a0f1-3ba4fa5de596 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.537466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 242/483] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:28.542775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5825c3-a7dd-41c5-bf1c-bdd43b7ea9a3 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.543062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5825c3-a7dd-41c5-bf1c-bdd43b7ea9a3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.543873 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5825c3-a7dd-41c5-bf1c-bdd43b7ea9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.588847 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7f5825c3-a7dd-41c5-bf1c-bdd43b7ea9a3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:28.590056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7f5825c3-a7dd-41c5-bf1c-bdd43b7ea9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:28.684710 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 242/484] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:28.799862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12bc86-795a-448f-a0f1-3ba4fa5de596 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:28.906631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12bc86-795a-448f-a0f1-3ba4fa5de596 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:28.933205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:28.942913 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0afc81-a2c6-49bb-ab8c-cc4fd63a4977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:28.943521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0afc81-a2c6-49bb-ab8c-cc4fd63a4977 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.943801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0afc81-a2c6-49bb-ab8c-cc4fd63a4977 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.944069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0afc81-a2c6-49bb-ab8c-cc4fd63a4977 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.955271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0afc81-a2c6-49bb-ab8c-cc4fd63a4977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-8a0afc81-a2c6-49bb-ab8c-cc4fd63a4977 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:28.960077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:28.960951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:28.961767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:28.961767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:28.961767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:28.961767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:28.961767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:28.961767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:28.961767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:28.961767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:28.961767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:28.961767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:28.961767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:28.961767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:28.961767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 243/485] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:28.967859 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5825c3-a7dd-41c5-bf1c-bdd43b7ea9a3 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.021640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:29.084423 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5825c3-a7dd-41c5-bf1c-bdd43b7ea9a3 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.138267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a5db0-cbb3-4bd2-8d00-8008a13ee2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.138884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a5db0-cbb3-4bd2-8d00-8008a13ee2a3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.139166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a5db0-cbb3-4bd2-8d00-8008a13ee2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.142359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a5db0-cbb3-4bd2-8d00-8008a13ee2a3 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.143073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a5db0-cbb3-4bd2-8d00-8008a13ee2a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:29.143391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a5db0-cbb3-4bd2-8d00-8008a13ee2a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.144895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a5db0-cbb3-4bd2-8d00-8008a13ee2a3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.154850 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0afc81-a2c6-49bb-ab8c-cc4fd63a4977 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.158415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 243/486] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:29.166754 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a5db0-cbb3-4bd2-8d00-8008a13ee2a3 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.312479 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0afc81-a2c6-49bb-ab8c-cc4fd63a4977 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.330601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a5db0-cbb3-4bd2-8d00-8008a13ee2a3 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.347858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:29.351118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac2c41c-aa07-4238-9e71-48225d5572d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.352207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2c41c-aa07-4238-9e71-48225d5572d3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.352650 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2c41c-aa07-4238-9e71-48225d5572d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.353073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac2c41c-aa07-4238-9e71-48225d5572d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.366734 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac2c41c-aa07-4238-9e71-48225d5572d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:29.372059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6e1ee-a49e-41cc-99b7-c1fea87f20a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-0ac2c41c-aa07-4238-9e71-48225d5572d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:29.372735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:29.373635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:29.374505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:29.374505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:29.374505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:29.374505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:29.374505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:29.374505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:29.374505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:29.374505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:29.374505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:29.374505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:29.374505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:29.374505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:29.374505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 244/487] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:29.375654 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef871b0-6d68-4b7c-86ef-90383c7dd8ca None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.375654 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef871b0-6d68-4b7c-86ef-90383c7dd8ca None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.375881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef871b0-6d68-4b7c-86ef-90383c7dd8ca None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.376447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6e1ee-a49e-41cc-99b7-c1fea87f20a9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.376910 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6e1ee-a49e-41cc-99b7-c1fea87f20a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.377070 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 245/488] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:29.377509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6e1ee-a49e-41cc-99b7-c1fea87f20a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.382570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef871b0-6d68-4b7c-86ef-90383c7dd8ca None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.392088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6e1ee-a49e-41cc-99b7-c1fea87f20a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.394649 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6e1ee-a49e-41cc-99b7-c1fea87f20a9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-3df6e1ee-a49e-41cc-99b7-c1fea87f20a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:29.418163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:29.419920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:29.420943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:29.420943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:29.420943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:29.424805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 244/489] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:29.430352 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc73dd-93bc-4e79-8f04-2afe3fe7ccda None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.430352 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc73dd-93bc-4e79-8f04-2afe3fe7ccda None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.430352 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc73dd-93bc-4e79-8f04-2afe3fe7ccda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.449512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-38dc73dd-93bc-4e79-8f04-2afe3fe7ccda None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:29.450118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:29.450666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-38dc73dd-93bc-4e79-8f04-2afe3fe7ccda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:29.515279 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef871b0-6d68-4b7c-86ef-90383c7dd8ca None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.532968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 245/490] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:29.534195 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc73dd-93bc-4e79-8f04-2afe3fe7ccda None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.552252 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68cfdb-ca8c-4ffa-bfc6-87aba6c9c0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.553016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68cfdb-ca8c-4ffa-bfc6-87aba6c9c0d1 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.553275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68cfdb-ca8c-4ffa-bfc6-87aba6c9c0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.553546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68cfdb-ca8c-4ffa-bfc6-87aba6c9c0d1 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.554282 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68cfdb-ca8c-4ffa-bfc6-87aba6c9c0d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:29.554580 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68cfdb-ca8c-4ffa-bfc6-87aba6c9c0d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.556052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68cfdb-ca8c-4ffa-bfc6-87aba6c9c0d1 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.566693 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 246/491] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:29.568526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc563301-adc6-4810-9a02-c0106100f172 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.568808 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc563301-adc6-4810-9a02-c0106100f172 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.569072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc563301-adc6-4810-9a02-c0106100f172 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.570923 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 247/492] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:29.571949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc563301-adc6-4810-9a02-c0106100f172 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.639292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:29.648554 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc73dd-93bc-4e79-8f04-2afe3fe7ccda None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.695050 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bb1d6-178d-4726-8767-f1287ed72e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.695659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bb1d6-178d-4726-8767-f1287ed72e07 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.695936 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bb1d6-178d-4726-8767-f1287ed72e07 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.696204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bb1d6-178d-4726-8767-f1287ed72e07 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.696824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc563301-adc6-4810-9a02-c0106100f172 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.696931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bb1d6-178d-4726-8767-f1287ed72e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:29.697179 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bb1d6-178d-4726-8767-f1287ed72e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.698642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bb1d6-178d-4726-8767-f1287ed72e07 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.706723 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:29.717886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 246/493] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:29.717886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446af85-cebd-47f1-bc6d-06c43ed425ce None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.717886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446af85-cebd-47f1-bc6d-06c43ed425ce None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.717886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446af85-cebd-47f1-bc6d-06c43ed425ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.739589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e446af85-cebd-47f1-bc6d-06c43ed425ce None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:29.740813 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e446af85-cebd-47f1-bc6d-06c43ed425ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:29.743394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4227ed-34bb-442e-9e6c-359ad40a486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.744328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4227ed-34bb-442e-9e6c-359ad40a486d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.744838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4227ed-34bb-442e-9e6c-359ad40a486d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.745300 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4227ed-34bb-442e-9e6c-359ad40a486d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.757656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4227ed-34bb-442e-9e6c-359ad40a486d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.760247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4227ed-34bb-442e-9e6c-359ad40a486d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-0e4227ed-34bb-442e-9e6c-359ad40a486d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:29.777563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:29.778572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:29.779520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:29.779520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:29.779520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:29.782202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 248/494] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:29.783243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4227ed-34bb-442e-9e6c-359ad40a486d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.826871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 247/495] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:29.827492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446af85-cebd-47f1-bc6d-06c43ed425ce None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.904359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4227ed-34bb-442e-9e6c-359ad40a486d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.925279 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:29.941440 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454ec7a-636e-4dfa-bb4f-6b2ede9e01d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.941826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446af85-cebd-47f1-bc6d-06c43ed425ce None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.942090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454ec7a-636e-4dfa-bb4f-6b2ede9e01d6 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.942364 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454ec7a-636e-4dfa-bb4f-6b2ede9e01d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.942629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454ec7a-636e-4dfa-bb4f-6b2ede9e01d6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.951436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454ec7a-636e-4dfa-bb4f-6b2ede9e01d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.952856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454ec7a-636e-4dfa-bb4f-6b2ede9e01d6 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-5454ec7a-636e-4dfa-bb4f-6b2ede9e01d6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:29.967427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:29.968335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:29.969240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:29.969240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:29.969240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:29.969240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:29.969240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:29.969240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:29.969240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:29.969240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:29.969240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:39:29.969240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:29.969240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 249/496] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:29.970570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454ec7a-636e-4dfa-bb4f-6b2ede9e01d6 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.983283 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33f478-f902-4aa2-8810-beba0c39fbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.983891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33f478-f902-4aa2-8810-beba0c39fbec None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.984164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33f478-f902-4aa2-8810-beba0c39fbec None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.984436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33f478-f902-4aa2-8810-beba0c39fbec None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.985076 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33f478-f902-4aa2-8810-beba0c39fbec None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:29.985373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33f478-f902-4aa2-8810-beba0c39fbec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.986759 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33f478-f902-4aa2-8810-beba0c39fbec None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.995928 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.002576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 248/497] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:30.002576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6da57e-192e-42a5-8f04-fc71ee9c0754 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.002576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6da57e-192e-42a5-8f04-fc71ee9c0754 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.002576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6da57e-192e-42a5-8f04-fc71ee9c0754 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.002576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 249/498] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:30.002576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6da57e-192e-42a5-8f04-fc71ee9c0754 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.086795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454ec7a-636e-4dfa-bb4f-6b2ede9e01d6 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.108964 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6da57e-192e-42a5-8f04-fc71ee9c0754 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.120495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a682f9-eff6-4d49-8407-012e4e87297a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.121116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a682f9-eff6-4d49-8407-012e4e87297a None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.121478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a682f9-eff6-4d49-8407-012e4e87297a None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.121774 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a682f9-eff6-4d49-8407-012e4e87297a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.131973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a682f9-eff6-4d49-8407-012e4e87297a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-e6a682f9-eff6-4d49-8407-012e4e87297a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:30.137147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:30.138182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:30.138985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:30.138985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:30.138985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:30.138985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:30.138985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:30.138985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:30.138985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:30.138985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:30.138985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:30.138985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:30.138985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:30.138985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:30.138985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 250/499] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:30.140004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a682f9-eff6-4d49-8407-012e4e87297a None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.142665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909b025-2563-40c2-ab98-b955bcc9c40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.143272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909b025-2563-40c2-ab98-b955bcc9c40b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.143551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909b025-2563-40c2-ab98-b955bcc9c40b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.143821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909b025-2563-40c2-ab98-b955bcc9c40b None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.144541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909b025-2563-40c2-ab98-b955bcc9c40b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:30.144846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909b025-2563-40c2-ab98-b955bcc9c40b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.146304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909b025-2563-40c2-ab98-b955bcc9c40b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.182119 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 250/500] 10.222.0.55 () {56 vars in 1137 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:30.183317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76703a4d-075f-4b85-acdc-b85c431729fd None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.183595 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76703a4d-075f-4b85-acdc-b85c431729fd None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.184394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76703a4d-075f-4b85-acdc-b85c431729fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.191164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.204023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-76703a4d-075f-4b85-acdc-b85c431729fd None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:30.205205 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-76703a4d-075f-4b85-acdc-b85c431729fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:30.225376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.244218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a682f9-eff6-4d49-8407-012e4e87297a None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.272253 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 251/501] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:30.273093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76703a4d-075f-4b85-acdc-b85c431729fd None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.297474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88764da8-2a85-4f2e-8c81-49718ad4ce79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.298179 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88764da8-2a85-4f2e-8c81-49718ad4ce79 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.298434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88764da8-2a85-4f2e-8c81-49718ad4ce79 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.298714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88764da8-2a85-4f2e-8c81-49718ad4ce79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.307902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88764da8-2a85-4f2e-8c81-49718ad4ce79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.309904 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88764da8-2a85-4f2e-8c81-49718ad4ce79 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-88764da8-2a85-4f2e-8c81-49718ad4ce79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:30.322980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:30.323855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:30.325321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:30.325321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:30.325321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:30.325321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:30.325321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:30.325321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:30.325321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:30.325321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:30.325321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:30.325321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:30.325321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 251/502] 10.222.0.55 () {56 vars in 1093 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:30.326410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88764da8-2a85-4f2e-8c81-49718ad4ce79 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.382415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76703a4d-075f-4b85-acdc-b85c431729fd None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.416575 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0764b26-096e-4398-8ed2-5212d0ce8ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.417206 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0764b26-096e-4398-8ed2-5212d0ce8ec9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.417485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0764b26-096e-4398-8ed2-5212d0ce8ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.417780 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0764b26-096e-4398-8ed2-5212d0ce8ec9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.420502 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.429871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0764b26-096e-4398-8ed2-5212d0ce8ec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.431319 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0764b26-096e-4398-8ed2-5212d0ce8ec9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.432728 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88764da8-2a85-4f2e-8c81-49718ad4ce79 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-d0764b26-096e-4398-8ed2-5212d0ce8ec9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:30.446648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:30.447589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:30.448538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:30.448538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:30.448538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:30.448538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 252/503] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:30.448538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0764b26-096e-4398-8ed2-5212d0ce8ec9 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.472965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663ae49-ce67-47e6-b658-4e8639892f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.473599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663ae49-ce67-47e6-b658-4e8639892f06 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.473875 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663ae49-ce67-47e6-b658-4e8639892f06 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.474146 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663ae49-ce67-47e6-b658-4e8639892f06 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.474809 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663ae49-ce67-47e6-b658-4e8639892f06 None admin] RBAC: Adding query filter params (name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:30.475112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663ae49-ce67-47e6-b658-4e8639892f06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.476462 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663ae49-ce67-47e6-b658-4e8639892f06 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.488737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 252/504] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:30.488737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663ae49-ce67-47e6-b658-4e8639892f06 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.488737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.554824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0764b26-096e-4398-8ed2-5212d0ce8ec9 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.588238 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663ae49-ce67-47e6-b658-4e8639892f06 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.588668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c0dbe-9f46-4cb3-8775-c5c99a8ab427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.589267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c0dbe-9f46-4cb3-8775-c5c99a8ab427 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.589546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c0dbe-9f46-4cb3-8775-c5c99a8ab427 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.589846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c0dbe-9f46-4cb3-8775-c5c99a8ab427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.621053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df87e82-13b7-460f-a4fd-81eada547828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.621682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df87e82-13b7-460f-a4fd-81eada547828 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.621989 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df87e82-13b7-460f-a4fd-81eada547828 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.622269 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df87e82-13b7-460f-a4fd-81eada547828 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.622997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df87e82-13b7-460f-a4fd-81eada547828 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:30.623305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df87e82-13b7-460f-a4fd-81eada547828 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.624744 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df87e82-13b7-460f-a4fd-81eada547828 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.641543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.654437 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c0dbe-9f46-4cb3-8775-c5c99a8ab427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.655888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c0dbe-9f46-4cb3-8775-c5c99a8ab427 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.660772 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 253/505] 10.222.0.55 () {56 vars in 1133 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:30.661604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df87e82-13b7-460f-a4fd-81eada547828 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.720067 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.755163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 253/506] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:30.755998 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c0dbe-9f46-4cb3-8775-c5c99a8ab427 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.763861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df87e82-13b7-460f-a4fd-81eada547828 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.796834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43891a-18d5-457c-bf0b-62a13a0e215d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.797436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43891a-18d5-457c-bf0b-62a13a0e215d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.797706 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43891a-18d5-457c-bf0b-62a13a0e215d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.798003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43891a-18d5-457c-bf0b-62a13a0e215d None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.798632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43891a-18d5-457c-bf0b-62a13a0e215d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:30.798929 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43891a-18d5-457c-bf0b-62a13a0e215d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.800285 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43891a-18d5-457c-bf0b-62a13a0e215d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.811329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 254/507] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:30.812305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43891a-18d5-457c-bf0b-62a13a0e215d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.842073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.856981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c0dbe-9f46-4cb3-8775-c5c99a8ab427 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.889379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179480d-7cd8-4ff8-b4b6-f2aa1eb2a778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.890022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179480d-7cd8-4ff8-b4b6-f2aa1eb2a778 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.890296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179480d-7cd8-4ff8-b4b6-f2aa1eb2a778 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.890563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179480d-7cd8-4ff8-b4b6-f2aa1eb2a778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.900278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179480d-7cd8-4ff8-b4b6-f2aa1eb2a778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.901664 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179480d-7cd8-4ff8-b4b6-f2aa1eb2a778 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.917765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43891a-18d5-457c-bf0b-62a13a0e215d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1179480d-7cd8-4ff8-b4b6-f2aa1eb2a778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:30.921024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:30.922134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:30.923228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:30.923228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:30.923228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:30.923228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 254/508] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:30.927922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179480d-7cd8-4ff8-b4b6-f2aa1eb2a778 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.940389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.955072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff5df0-db7a-4c14-8611-1ce64f8c3755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.955814 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff5df0-db7a-4c14-8611-1ce64f8c3755 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.956224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff5df0-db7a-4c14-8611-1ce64f8c3755 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.956521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff5df0-db7a-4c14-8611-1ce64f8c3755 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.968160 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff5df0-db7a-4c14-8611-1ce64f8c3755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-3bff5df0-db7a-4c14-8611-1ce64f8c3755 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:30.973384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:30.974361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:30.975178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:30.975178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:30.975178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:30.975178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:30.975178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:30.975178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:30.975178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:30.975178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:30.975178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:30.975178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:30.975178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:30.975178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:30.975178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 255/509] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:30.976232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff5df0-db7a-4c14-8611-1ce64f8c3755 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.031070 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179480d-7cd8-4ff8-b4b6-f2aa1eb2a778 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.063832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d086e-5346-484b-8d4f-bef9abfc7fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.064436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d086e-5346-484b-8d4f-bef9abfc7fa7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.064714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d086e-5346-484b-8d4f-bef9abfc7fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.064986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d086e-5346-484b-8d4f-bef9abfc7fa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.073354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:31.077421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff5df0-db7a-4c14-8611-1ce64f8c3755 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.111550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c1859-ae7d-49ff-83d7-fab1d81534ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.112169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c1859-ae7d-49ff-83d7-fab1d81534ab None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.112446 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c1859-ae7d-49ff-83d7-fab1d81534ab None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.112719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c1859-ae7d-49ff-83d7-fab1d81534ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.123464 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c1859-ae7d-49ff-83d7-fab1d81534ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.124899 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c1859-ae7d-49ff-83d7-fab1d81534ab None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.131824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d086e-5346-484b-8d4f-bef9abfc7fa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.133283 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d086e-5346-484b-8d4f-bef9abfc7fa7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-a56c1859-ae7d-49ff-83d7-fab1d81534ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:31.139041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:31.140110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:31.141212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:31.141212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:31.141212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:31.141212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 256/510] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:31.141212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c1859-ae7d-49ff-83d7-fab1d81534ab None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.172342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:31.232482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 255/511] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:31.233326 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d086e-5346-484b-8d4f-bef9abfc7fa7 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.245934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c1859-ae7d-49ff-83d7-fab1d81534ab None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.272025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:31.280418 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea3896-12eb-479e-935b-29286cc42fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.281017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea3896-12eb-479e-935b-29286cc42fcd None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.281297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea3896-12eb-479e-935b-29286cc42fcd None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.281571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dea3896-12eb-479e-935b-29286cc42fcd None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.282249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea3896-12eb-479e-935b-29286cc42fcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.283682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dea3896-12eb-479e-935b-29286cc42fcd None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.304504 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 257/512] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:39:31 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:31.305330 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea3896-12eb-479e-935b-29286cc42fcd None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.348562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d086e-5346-484b-8d4f-bef9abfc7fa7 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.386787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd841f-b19f-4a0d-8651-7f83289340fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.387462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd841f-b19f-4a0d-8651-7f83289340fe None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.387753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd841f-b19f-4a0d-8651-7f83289340fe None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.388027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd841f-b19f-4a0d-8651-7f83289340fe None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.388689 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd841f-b19f-4a0d-8651-7f83289340fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:31.388996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd841f-b19f-4a0d-8651-7f83289340fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.390486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd841f-b19f-4a0d-8651-7f83289340fe None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.407170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 256/513] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:31.407170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd841f-b19f-4a0d-8651-7f83289340fe None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.423683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:31.441098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dea3896-12eb-479e-935b-29286cc42fcd None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.483020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1702b-a1a7-48ea-8d91-de6128e18e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.483643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1702b-a1a7-48ea-8d91-de6128e18e8a None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.483844 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1702b-a1a7-48ea-8d91-de6128e18e8a None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.484118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1702b-a1a7-48ea-8d91-de6128e18e8a None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.484801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1702b-a1a7-48ea-8d91-de6128e18e8a None admin] RBAC: Adding query filter params (name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:31.485105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1702b-a1a7-48ea-8d91-de6128e18e8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.486513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1702b-a1a7-48ea-8d91-de6128e18e8a None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.496056 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 258/514] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:31.496862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1702b-a1a7-48ea-8d91-de6128e18e8a None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.521586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:31.526366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd841f-b19f-4a0d-8651-7f83289340fe None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.564181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a2835-84d2-481f-ba3c-7296ea9530f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.564935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a2835-84d2-481f-ba3c-7296ea9530f7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.565308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a2835-84d2-481f-ba3c-7296ea9530f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.565764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a2835-84d2-481f-ba3c-7296ea9530f7 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.566438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a2835-84d2-481f-ba3c-7296ea9530f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:31.566838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a2835-84d2-481f-ba3c-7296ea9530f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.568299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a2835-84d2-481f-ba3c-7296ea9530f7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.581482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 257/515] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:31.585844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a2835-84d2-481f-ba3c-7296ea9530f7 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.611433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1702b-a1a7-48ea-8d91-de6128e18e8a None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.647306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8a74f-e22e-4914-8a0a-48431fa2b6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.648129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8a74f-e22e-4914-8a0a-48431fa2b6c8 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.648584 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8a74f-e22e-4914-8a0a-48431fa2b6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.648924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8a74f-e22e-4914-8a0a-48431fa2b6c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.661143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8a74f-e22e-4914-8a0a-48431fa2b6c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.662768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c8a74f-e22e-4914-8a0a-48431fa2b6c8 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-92c8a74f-e22e-4914-8a0a-48431fa2b6c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:31.679178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:31.680294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:31.681214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:31.681214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:31.681214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:31.681214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 259/516] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:31.681499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8a74f-e22e-4914-8a0a-48431fa2b6c8 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.685066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:31.692550 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a2835-84d2-481f-ba3c-7296ea9530f7 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.731367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9035d54-fc13-49d1-8150-5dce2a94f32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.731990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9035d54-fc13-49d1-8150-5dce2a94f32f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.732266 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9035d54-fc13-49d1-8150-5dce2a94f32f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.732536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9035d54-fc13-49d1-8150-5dce2a94f32f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.739743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9035d54-fc13-49d1-8150-5dce2a94f32f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.741188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9035d54-fc13-49d1-8150-5dce2a94f32f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-e9035d54-fc13-49d1-8150-5dce2a94f32f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:31.753823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:31.754768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:31.755682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:31.755682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:31.755682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:31.755682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:31.755682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:31.755682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:31.755682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:31.755682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:31.755682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:39:31.755682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:31.759576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 258/517] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:31.760410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9035d54-fc13-49d1-8150-5dce2a94f32f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.780539 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:31.796541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8a74f-e22e-4914-8a0a-48431fa2b6c8 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.833043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139c7a0-16f2-43a7-82d1-f585d19d1eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.833868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139c7a0-16f2-43a7-82d1-f585d19d1eed None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.834257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139c7a0-16f2-43a7-82d1-f585d19d1eed None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.834634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139c7a0-16f2-43a7-82d1-f585d19d1eed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.846656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139c7a0-16f2-43a7-82d1-f585d19d1eed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.848244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139c7a0-16f2-43a7-82d1-f585d19d1eed None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-7139c7a0-16f2-43a7-82d1-f585d19d1eed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:31.863096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:31.864219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:31.865244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:31.865244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:31.865244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:31.870572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 260/518] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:31.870572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139c7a0-16f2-43a7-82d1-f585d19d1eed None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.877379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9035d54-fc13-49d1-8150-5dce2a94f32f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.918427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be43553-56ce-48c9-8cf9-435f08104095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.919754 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be43553-56ce-48c9-8cf9-435f08104095 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.920173 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be43553-56ce-48c9-8cf9-435f08104095 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.920683 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be43553-56ce-48c9-8cf9-435f08104095 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.927443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:31.928744 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be43553-56ce-48c9-8cf9-435f08104095 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.930476 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be43553-56ce-48c9-8cf9-435f08104095 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-5be43553-56ce-48c9-8cf9-435f08104095 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:31.942267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:31.943114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:31.944022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:31.944022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:31.944022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:31.944022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:31.944022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:31.944022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:31.944022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:31.944022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:31.944022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 259/519] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:31.945462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be43553-56ce-48c9-8cf9-435f08104095 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.982454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139c7a0-16f2-43a7-82d1-f585d19d1eed None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.018351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a3745-eb0d-461e-b86c-343549a22882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.018962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a3745-eb0d-461e-b86c-343549a22882 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.019241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a3745-eb0d-461e-b86c-343549a22882 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.019511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a3745-eb0d-461e-b86c-343549a22882 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.020156 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a3745-eb0d-461e-b86c-343549a22882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:32.020459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a3745-eb0d-461e-b86c-343549a22882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.021913 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a3745-eb0d-461e-b86c-343549a22882 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.026882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:32.040075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 261/520] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:32.041000 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a3745-eb0d-461e-b86c-343549a22882 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.052111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be43553-56ce-48c9-8cf9-435f08104095 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.089172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bc5c4-2485-4e8f-97a1-fdc4d124e53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.090088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bc5c4-2485-4e8f-97a1-fdc4d124e53d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.091013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bc5c4-2485-4e8f-97a1-fdc4d124e53d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.091441 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bc5c4-2485-4e8f-97a1-fdc4d124e53d None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.092442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bc5c4-2485-4e8f-97a1-fdc4d124e53d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:32.092756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bc5c4-2485-4e8f-97a1-fdc4d124e53d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.094322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bc5c4-2485-4e8f-97a1-fdc4d124e53d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.115801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 260/521] 10.222.0.55 () {56 vars in 1137 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:32.115801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bc5c4-2485-4e8f-97a1-fdc4d124e53d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.154164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a3745-eb0d-461e-b86c-343549a22882 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.168891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:32.191291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4014d66-e271-4f95-99b0-55547390de20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.191817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4014d66-e271-4f95-99b0-55547390de20 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.192094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4014d66-e271-4f95-99b0-55547390de20 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.192375 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4014d66-e271-4f95-99b0-55547390de20 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.193005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4014d66-e271-4f95-99b0-55547390de20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:32.193307 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4014d66-e271-4f95-99b0-55547390de20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.194947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4014d66-e271-4f95-99b0-55547390de20 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.210805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 262/522] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:32.211774 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4014d66-e271-4f95-99b0-55547390de20 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.219974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bc5c4-2485-4e8f-97a1-fdc4d124e53d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.256183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc026b6d-0a35-4b11-a282-779ca9130b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.256799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc026b6d-0a35-4b11-a282-779ca9130b2e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.257119 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc026b6d-0a35-4b11-a282-779ca9130b2e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.257369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc026b6d-0a35-4b11-a282-779ca9130b2e None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.258063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc026b6d-0a35-4b11-a282-779ca9130b2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:32.258370 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc026b6d-0a35-4b11-a282-779ca9130b2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.259823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc026b6d-0a35-4b11-a282-779ca9130b2e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.263676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:32.282615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 261/523] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:32.284769 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc026b6d-0a35-4b11-a282-779ca9130b2e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.320790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4014d66-e271-4f95-99b0-55547390de20 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.354304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a6bc0-a0fb-47f5-b1d8-97b8e7e4e426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.354928 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a6bc0-a0fb-47f5-b1d8-97b8e7e4e426 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.355205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a6bc0-a0fb-47f5-b1d8-97b8e7e4e426 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.355476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a6bc0-a0fb-47f5-b1d8-97b8e7e4e426 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.362612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a6bc0-a0fb-47f5-b1d8-97b8e7e4e426 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.364066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a6bc0-a0fb-47f5-b1d8-97b8e7e4e426 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-386a6bc0-a0fb-47f5-b1d8-97b8e7e4e426 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:32.376627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:32.377505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:32.378417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:32.378417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:32.378417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:32.378417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:32.378417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:32.378417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:32.378417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:32.378417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:32.378417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 263/524] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:32.378895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a6bc0-a0fb-47f5-b1d8-97b8e7e4e426 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.391113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc026b6d-0a35-4b11-a282-779ca9130b2e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.417868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:32.432950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a22239-469b-4bf6-aa11-05308111e9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.433566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a22239-469b-4bf6-aa11-05308111e9f5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.433909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a22239-469b-4bf6-aa11-05308111e9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.434184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a22239-469b-4bf6-aa11-05308111e9f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.492574 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a6bc0-a0fb-47f5-b1d8-97b8e7e4e426 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.498555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:32.506290 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a22239-469b-4bf6-aa11-05308111e9f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.508441 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a22239-469b-4bf6-aa11-05308111e9f5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.534001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba61437-0ef8-4ec3-8f69-6aabf477ef55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.535048 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba61437-0ef8-4ec3-8f69-6aabf477ef55 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.535341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba61437-0ef8-4ec3-8f69-6aabf477ef55 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.535623 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba61437-0ef8-4ec3-8f69-6aabf477ef55 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.544610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba61437-0ef8-4ec3-8f69-6aabf477ef55 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.546443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba61437-0ef8-4ec3-8f69-6aabf477ef55 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-6ba61437-0ef8-4ec3-8f69-6aabf477ef55 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:32.558138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:32.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:32.559981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:32.559981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:32.559981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:32.559981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:32.559981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:32.559981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:32.559981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:32.559981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:32.559981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:39:32.559981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:32.559981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 264/525] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:32.560552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba61437-0ef8-4ec3-8f69-6aabf477ef55 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.621795 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 262/526] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:32.621795 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a22239-469b-4bf6-aa11-05308111e9f5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.670935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:32.677838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba61437-0ef8-4ec3-8f69-6aabf477ef55 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.716677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:32.716983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996439fa-6fb4-4728-839f-1d8164436a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.718180 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996439fa-6fb4-4728-839f-1d8164436a16 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.718623 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996439fa-6fb4-4728-839f-1d8164436a16 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.719083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996439fa-6fb4-4728-839f-1d8164436a16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.720644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996439fa-6fb4-4728-839f-1d8164436a16 None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=4e27f5b81c704dbebbd6a83b2713e237) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:32.721198 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996439fa-6fb4-4728-839f-1d8164436a16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.723147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996439fa-6fb4-4728-839f-1d8164436a16 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.723331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a22239-469b-4bf6-aa11-05308111e9f5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.741766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 265/527] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/role_assignments?user.id=4e27f5b81c704dbebbd6a83b2713e237&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:32.741766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996439fa-6fb4-4728-839f-1d8164436a16 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.781249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b3780-cdf8-42af-a34a-d591bae4d402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.781815 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b3780-cdf8-42af-a34a-d591bae4d402 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.782168 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b3780-cdf8-42af-a34a-d591bae4d402 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.782444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b3780-cdf8-42af-a34a-d591bae4d402 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.783215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b3780-cdf8-42af-a34a-d591bae4d402 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:32.783528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b3780-cdf8-42af-a34a-d591bae4d402 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.785003 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b3780-cdf8-42af-a34a-d591bae4d402 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.801909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 263/528] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:32.803237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b3780-cdf8-42af-a34a-d591bae4d402 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.862934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996439fa-6fb4-4728-839f-1d8164436a16 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.908880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0b129-aa6f-4af7-aacd-c0f19e6b0cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.909890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0b129-aa6f-4af7-aacd-c0f19e6b0cec None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.910370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0b129-aa6f-4af7-aacd-c0f19e6b0cec None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.910836 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd0b129-aa6f-4af7-aacd-c0f19e6b0cec None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.911811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0b129-aa6f-4af7-aacd-c0f19e6b0cec None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:32.912207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0b129-aa6f-4af7-aacd-c0f19e6b0cec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.913794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd0b129-aa6f-4af7-aacd-c0f19e6b0cec None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.937772 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 266/529] 10.222.0.55 () {56 vars in 1139 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:32.938846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0b129-aa6f-4af7-aacd-c0f19e6b0cec None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.957640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b3780-cdf8-42af-a34a-d591bae4d402 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.967103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:32.995892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9919c5-4e90-4ca7-a98c-ea96cbe0cb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.996703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9919c5-4e90-4ca7-a98c-ea96cbe0cb3c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.997090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9919c5-4e90-4ca7-a98c-ea96cbe0cb3c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.997541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9919c5-4e90-4ca7-a98c-ea96cbe0cb3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.009505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9919c5-4e90-4ca7-a98c-ea96cbe0cb3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:33.011113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9919c5-4e90-4ca7-a98c-ea96cbe0cb3c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-aa9919c5-4e90-4ca7-a98c-ea96cbe0cb3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:33.028979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:33.030103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:33.030973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:33.030973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:33.030973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:33.030973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 264/530] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:33.031197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9919c5-4e90-4ca7-a98c-ea96cbe0cb3c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:33.043522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:33.078590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd0b129-aa6f-4af7-aacd-c0f19e6b0cec None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:33.116752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c216c-d945-4df8-86cf-c4faf2f9c04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:33.117621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c216c-d945-4df8-86cf-c4faf2f9c04d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.117752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c216c-d945-4df8-86cf-c4faf2f9c04d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.118079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c216c-d945-4df8-86cf-c4faf2f9c04d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.118872 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c216c-d945-4df8-86cf-c4faf2f9c04d None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=6ffa73f81539472391983c51447af1b6) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:33.119249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c216c-d945-4df8-86cf-c4faf2f9c04d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:33.120924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c216c-d945-4df8-86cf-c4faf2f9c04d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:33.127738 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 267/531] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/role_assignments?user.id=6ffa73f81539472391983c51447af1b6&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:33.128743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c216c-d945-4df8-86cf-c4faf2f9c04d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:33.164113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9919c5-4e90-4ca7-a98c-ea96cbe0cb3c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:33.201428 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ebc5f-c043-491a-8a29-8b5223bb73a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:33.202174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ebc5f-c043-491a-8a29-8b5223bb73a0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.202517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ebc5f-c043-491a-8a29-8b5223bb73a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.202854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ebc5f-c043-491a-8a29-8b5223bb73a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.238244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:33.266088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c216c-d945-4df8-86cf-c4faf2f9c04d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:33.278956 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ebc5f-c043-491a-8a29-8b5223bb73a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:33.280396 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ebc5f-c043-491a-8a29-8b5223bb73a0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:33.292527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:33.313517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab6c11-c7b3-44d5-aac9-000691c0eb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:33.315680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab6c11-c7b3-44d5-aac9-000691c0eb5b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.316333 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab6c11-c7b3-44d5-aac9-000691c0eb5b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.318114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab6c11-c7b3-44d5-aac9-000691c0eb5b None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.320777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab6c11-c7b3-44d5-aac9-000691c0eb5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:33.321447 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab6c11-c7b3-44d5-aac9-000691c0eb5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:33.324203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab6c11-c7b3-44d5-aac9-000691c0eb5b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:33.354101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 268/532] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:33.360015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab6c11-c7b3-44d5-aac9-000691c0eb5b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:33.474485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 265/533] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:33.481651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab6c11-c7b3-44d5-aac9-000691c0eb5b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:33.482804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ebc5f-c043-491a-8a29-8b5223bb73a0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:33.519566 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84120671-dfb3-49cd-9987-706a6342bb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:33.520823 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84120671-dfb3-49cd-9987-706a6342bb9f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.521243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84120671-dfb3-49cd-9987-706a6342bb9f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.521667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84120671-dfb3-49cd-9987-706a6342bb9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.529555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84120671-dfb3-49cd-9987-706a6342bb9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:33.531168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84120671-dfb3-49cd-9987-706a6342bb9f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:33.533945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-84120671-dfb3-49cd-9987-706a6342bb9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:33.546193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:33.547266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:33.548296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:33.548296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:33.548296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:33.548296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:33.548296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:33.548296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:33.548296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:33.548296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:33.550073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 269/534] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:33.551602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8911ec-1570-46c0-9912-a32341995c8c None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.552024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8911ec-1570-46c0-9912-a32341995c8c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.552447 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8911ec-1570-46c0-9912-a32341995c8c None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.554018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 270/535] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:33.555903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8911ec-1570-46c0-9912-a32341995c8c None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:33.609771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ebc5f-c043-491a-8a29-8b5223bb73a0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:33.619371 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:33.648823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d932301f-7e6a-4b8b-972f-555f7b8052f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:33.649639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932301f-7e6a-4b8b-972f-555f7b8052f2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.650131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932301f-7e6a-4b8b-972f-555f7b8052f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.650406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d932301f-7e6a-4b8b-972f-555f7b8052f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.661196 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932301f-7e6a-4b8b-972f-555f7b8052f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:33.662723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d932301f-7e6a-4b8b-972f-555f7b8052f2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:33.675783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8911ec-1570-46c0-9912-a32341995c8c None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-d932301f-7e6a-4b8b-972f-555f7b8052f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:33.679006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:33.680105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:33.681157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:33.681157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:33.681157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:33.686348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 266/536] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:33.686348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e195ba-5355-42ec-af83-c619c3e525ff None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.686348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e195ba-5355-42ec-af83-c619c3e525ff None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.686348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e195ba-5355-42ec-af83-c619c3e525ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.703198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f3e195ba-5355-42ec-af83-c619c3e525ff None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:33.704539 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f3e195ba-5355-42ec-af83-c619c3e525ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:33.724354 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ec746-eefb-4815-94bd-6bc57b74c386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:33.725167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ec746-eefb-4815-94bd-6bc57b74c386 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.725631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ec746-eefb-4815-94bd-6bc57b74c386 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.726205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451ec746-eefb-4815-94bd-6bc57b74c386 None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.727097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ec746-eefb-4815-94bd-6bc57b74c386 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:33.727577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ec746-eefb-4815-94bd-6bc57b74c386 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:33.729300 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451ec746-eefb-4815-94bd-6bc57b74c386 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:33.749378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 271/537] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:33.750630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ec746-eefb-4815-94bd-6bc57b74c386 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:33.775268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 267/538] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:33.776355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e195ba-5355-42ec-af83-c619c3e525ff None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:33.817665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:33.863875 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451ec746-eefb-4815-94bd-6bc57b74c386 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:33.876282 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:33.896735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e195ba-5355-42ec-af83-c619c3e525ff None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:33.902263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62578060-2e54-4eaa-99d5-1548295446df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:33.903093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62578060-2e54-4eaa-99d5-1548295446df None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.903509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62578060-2e54-4eaa-99d5-1548295446df None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.903907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62578060-2e54-4eaa-99d5-1548295446df None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.904693 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62578060-2e54-4eaa-99d5-1548295446df None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:33.905230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62578060-2e54-4eaa-99d5-1548295446df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:33.906746 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62578060-2e54-4eaa-99d5-1548295446df None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:33.920012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 272/539] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:33.925459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62578060-2e54-4eaa-99d5-1548295446df None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:33.959994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4c456-e61c-43ed-8a8d-79a4fd346153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:33.960676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4c456-e61c-43ed-8a8d-79a4fd346153 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.961048 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4c456-e61c-43ed-8a8d-79a4fd346153 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.961418 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4c456-e61c-43ed-8a8d-79a4fd346153 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.962313 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4c456-e61c-43ed-8a8d-79a4fd346153 None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=ff93243f0c4c48d1924ea04a87a0e7c1) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:33.962719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4c456-e61c-43ed-8a8d-79a4fd346153 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:33.964316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f4c456-e61c-43ed-8a8d-79a4fd346153 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:33.975586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 268/540] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:33 2022] GET /identity/v3/role_assignments?user.id=ff93243f0c4c48d1924ea04a87a0e7c1&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:34.036666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62578060-2e54-4eaa-99d5-1548295446df None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.058225 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:34.074212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f4627-d3ed-473f-82f0-ba33ab5d0925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.075707 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f4627-d3ed-473f-82f0-ba33ab5d0925 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.076127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f4627-d3ed-473f-82f0-ba33ab5d0925 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.076540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f4627-d3ed-473f-82f0-ba33ab5d0925 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.084445 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f4627-d3ed-473f-82f0-ba33ab5d0925 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.086211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f4627-d3ed-473f-82f0-ba33ab5d0925 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.092388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4c456-e61c-43ed-8a8d-79a4fd346153 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-131f4627-d3ed-473f-82f0-ba33ab5d0925 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:34.099540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:34.100422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:34.101297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:34.101297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:34.101297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:34.101297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:34.101297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:34.101297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:34.101297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:34.101297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:34.101297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 273/541] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:34.113176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f4627-d3ed-473f-82f0-ba33ab5d0925 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:34.225042 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4c456-e61c-43ed-8a8d-79a4fd346153 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.296578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07c558-3001-4672-b9d2-fb805725ba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.297833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07c558-3001-4672-b9d2-fb805725ba41 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.299036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07c558-3001-4672-b9d2-fb805725ba41 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.300868 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07c558-3001-4672-b9d2-fb805725ba41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.317810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f4627-d3ed-473f-82f0-ba33ab5d0925 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.342370 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07c558-3001-4672-b9d2-fb805725ba41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-0c07c558-3001-4672-b9d2-fb805725ba41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:34.347770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:34.348669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:34.349459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:34.349459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:34.349459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:34.349459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:34.349459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:34.349459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:34.349459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:34.349459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:34.349459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:34.349459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:34.349459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:34.349459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:34.349459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 269/542] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:34.357145 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f1abf-2c13-4adc-8a25-70ddf10f8deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.358105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f1abf-2c13-4adc-8a25-70ddf10f8deb None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.358561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f1abf-2c13-4adc-8a25-70ddf10f8deb None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.358663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07c558-3001-4672-b9d2-fb805725ba41 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:34.359218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f1abf-2c13-4adc-8a25-70ddf10f8deb None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.360110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f1abf-2c13-4adc-8a25-70ddf10f8deb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:34.360588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f1abf-2c13-4adc-8a25-70ddf10f8deb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.362233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f1abf-2c13-4adc-8a25-70ddf10f8deb None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.376094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 274/543] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:34.382136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f1abf-2c13-4adc-8a25-70ddf10f8deb None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:34.395066 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:34.430257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:34.491011 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f1abf-2c13-4adc-8a25-70ddf10f8deb None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.530924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb61d-64b9-41d2-b553-c2edee61326e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.532343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb61d-64b9-41d2-b553-c2edee61326e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.532827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb61d-64b9-41d2-b553-c2edee61326e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.533440 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cb61d-64b9-41d2-b553-c2edee61326e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.534195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2cb61d-64b9-41d2-b553-c2edee61326e None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=531ebc79bf424e9b8762b2c7ec7fe8d0) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:34.534548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2cb61d-64b9-41d2-b553-c2edee61326e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.536003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2cb61d-64b9-41d2-b553-c2edee61326e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.542120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 275/544] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:34 2022] GET /identity/v3/role_assignments?user.id=531ebc79bf424e9b8762b2c7ec7fe8d0&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:34.598539 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07c558-3001-4672-b9d2-fb805725ba41 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.656100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d90d1-8c2b-4633-b6eb-825d0333bcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.656724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d90d1-8c2b-4633-b6eb-825d0333bcf0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.656990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d90d1-8c2b-4633-b6eb-825d0333bcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.657266 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d90d1-8c2b-4633-b6eb-825d0333bcf0 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.657963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d90d1-8c2b-4633-b6eb-825d0333bcf0 None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:34.658268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d90d1-8c2b-4633-b6eb-825d0333bcf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.659659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d90d1-8c2b-4633-b6eb-825d0333bcf0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.669389 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 270/545] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:34.675295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb61d-64b9-41d2-b553-c2edee61326e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:34.832625 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cb61d-64b9-41d2-b553-c2edee61326e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.835975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:34.874177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a47c05-7e23-47f4-aa8d-b48745655b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.874815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a47c05-7e23-47f4-aa8d-b48745655b91 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.875094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a47c05-7e23-47f4-aa8d-b48745655b91 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.875369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a47c05-7e23-47f4-aa8d-b48745655b91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.888125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a47c05-7e23-47f4-aa8d-b48745655b91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.889578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a47c05-7e23-47f4-aa8d-b48745655b91 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-13a47c05-7e23-47f4-aa8d-b48745655b91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:34.917616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:34.918637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:34.920123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:34.920123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:34.920123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:34.920123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 276/546] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:34.924482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d90d1-8c2b-4633-b6eb-825d0333bcf0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:35.035165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:35.060737 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d90d1-8c2b-4633-b6eb-825d0333bcf0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:35.099706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a50122-9c78-4841-9606-7512362bced9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:35.100243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a50122-9c78-4841-9606-7512362bced9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:35.100522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a50122-9c78-4841-9606-7512362bced9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:35.100792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a50122-9c78-4841-9606-7512362bced9 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:35.101429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a50122-9c78-4841-9606-7512362bced9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:35.101749 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a50122-9c78-4841-9606-7512362bced9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:35.103115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a50122-9c78-4841-9606-7512362bced9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:35.120607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 271/547] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:35.127191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a47c05-7e23-47f4-aa8d-b48745655b91 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:35.236320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcda82d-b80a-43b4-b5cb-18bc0b754b33 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:35.236624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcda82d-b80a-43b4-b5cb-18bc0b754b33 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:35.236891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcda82d-b80a-43b4-b5cb-18bc0b754b33 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:35.238465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 272/548] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:35.243191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258ce35-7865-4834-a0d9-e5f0e79286ae None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:35.243474 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258ce35-7865-4834-a0d9-e5f0e79286ae None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:35.243751 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258ce35-7865-4834-a0d9-e5f0e79286ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:35.262295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c258ce35-7865-4834-a0d9-e5f0e79286ae None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:35.263342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a47c05-7e23-47f4-aa8d-b48745655b91 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:35.263527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c258ce35-7865-4834-a0d9-e5f0e79286ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:35.308377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe980abc-e6f8-4024-8e87-fec0772211fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:35.309432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe980abc-e6f8-4024-8e87-fec0772211fa None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:35.310574 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe980abc-e6f8-4024-8e87-fec0772211fa None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:35.312235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe980abc-e6f8-4024-8e87-fec0772211fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:35.327595 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe980abc-e6f8-4024-8e87-fec0772211fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:35.329829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe980abc-e6f8-4024-8e87-fec0772211fa None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-fe980abc-e6f8-4024-8e87-fec0772211fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:35.349045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:35.350555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:35.351645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:35.351645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:35.351645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:35.351645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:35.351645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:35.351645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:35.351645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:35.351645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:35.351645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:35.351645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:35.351645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 277/549] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:35.356682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe980abc-e6f8-4024-8e87-fec0772211fa None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:35.384321 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 273/550] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:39:35.393705 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:35.482996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe980abc-e6f8-4024-8e87-fec0772211fa None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:35.517827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd6c3a-d9f8-4f56-a9fb-948778c4816d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:35.518439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd6c3a-d9f8-4f56-a9fb-948778c4816d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:35.518716 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd6c3a-d9f8-4f56-a9fb-948778c4816d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:35.518988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd6c3a-d9f8-4f56-a9fb-948778c4816d None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:35.519727 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd6c3a-d9f8-4f56-a9fb-948778c4816d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:35.520033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd6c3a-d9f8-4f56-a9fb-948778c4816d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:35.521511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fd6c3a-d9f8-4f56-a9fb-948778c4816d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:35.540968 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 278/551] 10.222.0.55 () {56 vars in 1135 bytes} [Thu Sep 8 13:39:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:35.550845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258ce35-7865-4834-a0d9-e5f0e79286ae None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:35.648424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:35.686279 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258ce35-7865-4834-a0d9-e5f0e79286ae None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:35.722884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d1bf7-1043-4ab5-8d4b-6060b03f7844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:35.723838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d1bf7-1043-4ab5-8d4b-6060b03f7844 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:35.724294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d1bf7-1043-4ab5-8d4b-6060b03f7844 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:35.724748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d1bf7-1043-4ab5-8d4b-6060b03f7844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:35.803555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d1bf7-1043-4ab5-8d4b-6060b03f7844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:35.805307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d1bf7-1043-4ab5-8d4b-6060b03f7844 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:35.897209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:35.912913 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd6c3a-d9f8-4f56-a9fb-948778c4816d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:35.942903 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 274/552] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:35.950133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d1bf7-1043-4ab5-8d4b-6060b03f7844 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.040985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:36.083135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd6c3a-d9f8-4f56-a9fb-948778c4816d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.126613 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22981548-27c7-4cec-b21c-3d74806e50d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.127297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22981548-27c7-4cec-b21c-3d74806e50d4 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.127828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22981548-27c7-4cec-b21c-3d74806e50d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.128092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22981548-27c7-4cec-b21c-3d74806e50d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.138312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d1bf7-1043-4ab5-8d4b-6060b03f7844 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.141297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22981548-27c7-4cec-b21c-3d74806e50d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-22981548-27c7-4cec-b21c-3d74806e50d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:36.147606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:36.148649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:36.149941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:36.149941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:36.149941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:36.149941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:36.149941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:36.149941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:36.149941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:36.149941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:36.149941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:36.149941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:36.149941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:36.149941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:36.149941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 279/553] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:36.156748 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22981548-27c7-4cec-b21c-3d74806e50d4 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.207968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c496b0-6d37-444f-8fc1-b6c283fbfec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.209020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c496b0-6d37-444f-8fc1-b6c283fbfec3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.209303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c496b0-6d37-444f-8fc1-b6c283fbfec3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.210031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c496b0-6d37-444f-8fc1-b6c283fbfec3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.221511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c496b0-6d37-444f-8fc1-b6c283fbfec3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.223128 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c496b0-6d37-444f-8fc1-b6c283fbfec3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-81c496b0-6d37-444f-8fc1-b6c283fbfec3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:36.238250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:36.239307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:36.240638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:36.240638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:36.240638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:36.240638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 275/554] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:36.240872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9727e39-3b30-476e-911a-8441a8d1db3f None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.241162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9727e39-3b30-476e-911a-8441a8d1db3f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.241314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9727e39-3b30-476e-911a-8441a8d1db3f None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.247270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 276/555] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:36.248032 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9727e39-3b30-476e-911a-8441a8d1db3f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.267646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:36.283228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22981548-27c7-4cec-b21c-3d74806e50d4 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.325159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436f324-ac04-41c4-9529-eb91a1d899d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.325811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436f324-ac04-41c4-9529-eb91a1d899d7 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.326092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436f324-ac04-41c4-9529-eb91a1d899d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.326579 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436f324-ac04-41c4-9529-eb91a1d899d7 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.327615 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436f324-ac04-41c4-9529-eb91a1d899d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:36.327615 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436f324-ac04-41c4-9529-eb91a1d899d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.329401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436f324-ac04-41c4-9529-eb91a1d899d7 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.336440 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:36.342367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 280/556] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:36.344046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a529de6-c1a9-48ec-bec1-8ecc4d6f3e2e None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.344552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a529de6-c1a9-48ec-bec1-8ecc4d6f3e2e None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.345907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a529de6-c1a9-48ec-bec1-8ecc4d6f3e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.368476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4a529de6-c1a9-48ec-bec1-8ecc4d6f3e2e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:36.368702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9727e39-3b30-476e-911a-8441a8d1db3f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.370439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4a529de6-c1a9-48ec-bec1-8ecc4d6f3e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:36.404379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067c055-bf9e-4ddd-93c5-aaebeaabfb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.404988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067c055-bf9e-4ddd-93c5-aaebeaabfb43 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.405261 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067c055-bf9e-4ddd-93c5-aaebeaabfb43 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.405571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067c055-bf9e-4ddd-93c5-aaebeaabfb43 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.406282 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067c055-bf9e-4ddd-93c5-aaebeaabfb43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:36.406534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067c055-bf9e-4ddd-93c5-aaebeaabfb43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.407940 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067c055-bf9e-4ddd-93c5-aaebeaabfb43 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.426671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 277/557] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:36.427963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067c055-bf9e-4ddd-93c5-aaebeaabfb43 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.456726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 281/558] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:36.457869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a529de6-c1a9-48ec-bec1-8ecc4d6f3e2e None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.519085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:36.551456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067c055-bf9e-4ddd-93c5-aaebeaabfb43 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.583201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a529de6-c1a9-48ec-bec1-8ecc4d6f3e2e None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.609096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197f099-b875-40c6-89a0-f50375f42c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.610299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197f099-b875-40c6-89a0-f50375f42c15 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.610765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197f099-b875-40c6-89a0-f50375f42c15 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.611579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197f099-b875-40c6-89a0-f50375f42c15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.613544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:36.623237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a4dce-eefd-4327-a85d-58d28bacaf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.624032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a4dce-eefd-4327-a85d-58d28bacaf58 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.624376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a4dce-eefd-4327-a85d-58d28bacaf58 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.624754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a4dce-eefd-4327-a85d-58d28bacaf58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.628442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197f099-b875-40c6-89a0-f50375f42c15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.630126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197f099-b875-40c6-89a0-f50375f42c15 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.632037 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a4dce-eefd-4327-a85d-58d28bacaf58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.633635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6a4dce-eefd-4327-a85d-58d28bacaf58 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-5a6a4dce-eefd-4327-a85d-58d28bacaf58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:36.645935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:36.646851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:36.647750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:36.647750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:36.647750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:36.647750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:36.647750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:36.647750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:36.647750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:36.647750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-e197f099-b875-40c6-89a0-f50375f42c15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:36.650951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:36.651945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:36.653477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:36.653477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:36.653477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:36.653648 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 282/559] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:36.654145 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69686e1-d58d-4c39-bb62-14a267aa8f49 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.654246 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69686e1-d58d-4c39-bb62-14a267aa8f49 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.654439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69686e1-d58d-4c39-bb62-14a267aa8f49 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.655824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 283/560] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:36.656355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69686e1-d58d-4c39-bb62-14a267aa8f49 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.659272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 278/561] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:36.664817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14c26f-e569-4093-87d9-050d74a8c063 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.664817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14c26f-e569-4093-87d9-050d74a8c063 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.664817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14c26f-e569-4093-87d9-050d74a8c063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.681723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4c14c26f-e569-4093-87d9-050d74a8c063 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:36.682932 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4c14c26f-e569-4093-87d9-050d74a8c063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:36.762051 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 279/562] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:39:36.762051 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14c26f-e569-4093-87d9-050d74a8c063 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.807621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69686e1-d58d-4c39-bb62-14a267aa8f49 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.825314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:36.855210 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739a67d-b80d-4a3e-af66-709d7dea1c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.856220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739a67d-b80d-4a3e-af66-709d7dea1c3e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.856498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739a67d-b80d-4a3e-af66-709d7dea1c3e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.856767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b739a67d-b80d-4a3e-af66-709d7dea1c3e None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.857514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739a67d-b80d-4a3e-af66-709d7dea1c3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:36.857844 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739a67d-b80d-4a3e-af66-709d7dea1c3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.859809 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b739a67d-b80d-4a3e-af66-709d7dea1c3e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.872171 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14c26f-e569-4093-87d9-050d74a8c063 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.881724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 284/563] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:36.882622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739a67d-b80d-4a3e-af66-709d7dea1c3e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.928486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead0489-42ef-4436-8700-0303e77fd760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.929518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead0489-42ef-4436-8700-0303e77fd760 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.931682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead0489-42ef-4436-8700-0303e77fd760 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.932104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead0489-42ef-4436-8700-0303e77fd760 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.935462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead0489-42ef-4436-8700-0303e77fd760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:36.935925 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead0489-42ef-4436-8700-0303e77fd760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.937508 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead0489-42ef-4436-8700-0303e77fd760 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.946801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:36.954121 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 280/564] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:36.955189 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead0489-42ef-4436-8700-0303e77fd760 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.014043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b739a67d-b80d-4a3e-af66-709d7dea1c3e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.058149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46311722-0fb5-4b19-a6d2-eca85d87d149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.059247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46311722-0fb5-4b19-a6d2-eca85d87d149 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.059715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46311722-0fb5-4b19-a6d2-eca85d87d149 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.060113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46311722-0fb5-4b19-a6d2-eca85d87d149 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.071083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead0489-42ef-4436-8700-0303e77fd760 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.071216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46311722-0fb5-4b19-a6d2-eca85d87d149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-46311722-0fb5-4b19-a6d2-eca85d87d149 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:37.076007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:37.076953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:37.078245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:37.078245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:37.078245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:37.078245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:37.078245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:37.078245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:37.078245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:37.078245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:37.078245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:37.078245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:37.078245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:37.078245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:37.078245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 285/565] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:37.078995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46311722-0fb5-4b19-a6d2-eca85d87d149 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.096497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:37.112104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b79ed-0505-4bb5-ae47-2e39dced36e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.112741 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b79ed-0505-4bb5-ae47-2e39dced36e8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.113022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b79ed-0505-4bb5-ae47-2e39dced36e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.113293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b79ed-0505-4bb5-ae47-2e39dced36e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.114036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b79ed-0505-4bb5-ae47-2e39dced36e8 None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=4e5d73e8daa84975bc21e41e3cbef9cb) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:37.114345 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b79ed-0505-4bb5-ae47-2e39dced36e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.115766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b79ed-0505-4bb5-ae47-2e39dced36e8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.121325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 281/566] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:36 2022] GET /identity/v3/role_assignments?user.id=4e5d73e8daa84975bc21e41e3cbef9cb&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:37.122149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b79ed-0505-4bb5-ae47-2e39dced36e8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.221439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46311722-0fb5-4b19-a6d2-eca85d87d149 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.232891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:37.242245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b79ed-0505-4bb5-ae47-2e39dced36e8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.275161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991daf0-a66d-47b1-9fc8-ed9aebe877cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.275936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991daf0-a66d-47b1-9fc8-ed9aebe877cf None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.276318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991daf0-a66d-47b1-9fc8-ed9aebe877cf None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.276599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991daf0-a66d-47b1-9fc8-ed9aebe877cf None admin] PATH_INFO: `/v3/users/glance` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.281950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0425e-2670-49f2-9e86-4248cecc89c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.282600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0425e-2670-49f2-9e86-4248cecc89c4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.282877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0425e-2670-49f2-9e86-4248cecc89c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.283149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0425e-2670-49f2-9e86-4248cecc89c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.285498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991daf0-a66d-47b1-9fc8-ed9aebe877cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.287428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991daf0-a66d-47b1-9fc8-ed9aebe877cf None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.294767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0425e-2670-49f2-9e86-4248cecc89c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-3fd0425e-2670-49f2-9e86-4248cecc89c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:37.300089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:37.301020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:37.302268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:37.302268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:37.302268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:37.302268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:37.302268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:37.302268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:37.302268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:37.302268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:37.302268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:37.302268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:37.302268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:37.302268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-c991daf0-a66d-47b1-9fc8-ed9aebe877cf None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:37.302911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:37.304753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:37.305708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:37.305708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:37.305708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:37.305708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:37.305708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:37.305708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:37.305708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:37.305708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:37.305708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:39:37.305708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:37.306291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 286/568] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:37.306359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 282/568] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:37.307177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991daf0-a66d-47b1-9fc8-ed9aebe877cf None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.307599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b95794-1bfc-4bd8-876d-012f88398451 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.307893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b95794-1bfc-4bd8-876d-012f88398451 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.308215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b95794-1bfc-4bd8-876d-012f88398451 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.309713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 283/569] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:37.310505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b95794-1bfc-4bd8-876d-012f88398451 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.369832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:37.423462 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991daf0-a66d-47b1-9fc8-ed9aebe877cf None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.428404 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b95794-1bfc-4bd8-876d-012f88398451 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.460824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80bfd3-4a03-47e7-9112-b7b427a6027b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.461433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80bfd3-4a03-47e7-9112-b7b427a6027b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.461708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80bfd3-4a03-47e7-9112-b7b427a6027b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.462045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80bfd3-4a03-47e7-9112-b7b427a6027b None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.462675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80bfd3-4a03-47e7-9112-b7b427a6027b None admin] RBAC: Adding query filter params (name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:37.462975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80bfd3-4a03-47e7-9112-b7b427a6027b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.464393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80bfd3-4a03-47e7-9112-b7b427a6027b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.466736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b69256-96cf-43ce-bf26-6be8ff70517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.467356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69256-96cf-43ce-bf26-6be8ff70517c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.467638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69256-96cf-43ce-bf26-6be8ff70517c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.467916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b69256-96cf-43ce-bf26-6be8ff70517c None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.468644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b69256-96cf-43ce-bf26-6be8ff70517c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:37.468952 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b69256-96cf-43ce-bf26-6be8ff70517c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.470409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b69256-96cf-43ce-bf26-6be8ff70517c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.474665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 287/570] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:37.475688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80bfd3-4a03-47e7-9112-b7b427a6027b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.488630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 284/571] 10.222.0.55 () {56 vars in 1137 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:37.489895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908a52b-908d-4176-8a98-894119c4c62f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.490180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908a52b-908d-4176-8a98-894119c4c62f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.490453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908a52b-908d-4176-8a98-894119c4c62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.508019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c908a52b-908d-4176-8a98-894119c4c62f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:37.509425 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c908a52b-908d-4176-8a98-894119c4c62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:37.514198 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:37.581348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 285/572] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:37.582333 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908a52b-908d-4176-8a98-894119c4c62f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.598532 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80bfd3-4a03-47e7-9112-b7b427a6027b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.630163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:37.641239 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf563c-73fa-4337-8c2f-d645a6d79f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.642213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf563c-73fa-4337-8c2f-d645a6d79f63 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.642694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf563c-73fa-4337-8c2f-d645a6d79f63 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.643579 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf563c-73fa-4337-8c2f-d645a6d79f63 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.645265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf563c-73fa-4337-8c2f-d645a6d79f63 None admin] RBAC: Adding query filter params (name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:37.645758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf563c-73fa-4337-8c2f-d645a6d79f63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.648031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf563c-73fa-4337-8c2f-d645a6d79f63 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.659996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 288/573] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:37.661015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf563c-73fa-4337-8c2f-d645a6d79f63 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.706419 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908a52b-908d-4176-8a98-894119c4c62f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.742293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab135d1b-491c-41ec-bb10-bb6a74967aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.743077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab135d1b-491c-41ec-bb10-bb6a74967aac None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.743534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab135d1b-491c-41ec-bb10-bb6a74967aac None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.743973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab135d1b-491c-41ec-bb10-bb6a74967aac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.754843 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab135d1b-491c-41ec-bb10-bb6a74967aac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.756465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab135d1b-491c-41ec-bb10-bb6a74967aac None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.771617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-ab135d1b-491c-41ec-bb10-bb6a74967aac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:37.773167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:37.774562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:37.775670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:37.775670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:37.775670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:37.776768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 286/574] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:37.777986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab135d1b-491c-41ec-bb10-bb6a74967aac None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.786150 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf563c-73fa-4337-8c2f-d645a6d79f63 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.825920 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214a039-3a3c-4c7a-b1eb-3e66663adfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.826802 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214a039-3a3c-4c7a-b1eb-3e66663adfdd None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.827302 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214a039-3a3c-4c7a-b1eb-3e66663adfdd None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.827778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214a039-3a3c-4c7a-b1eb-3e66663adfdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.840912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214a039-3a3c-4c7a-b1eb-3e66663adfdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.842678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214a039-3a3c-4c7a-b1eb-3e66663adfdd None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-a214a039-3a3c-4c7a-b1eb-3e66663adfdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:37.858820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:37.861131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:37.864220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:37.864220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:37.864220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:37.864220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 289/575] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:37.864220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214a039-3a3c-4c7a-b1eb-3e66663adfdd None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.880757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:37.901967 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab135d1b-491c-41ec-bb10-bb6a74967aac None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.943043 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52229628-28d7-4216-b1f9-7ed2a9b3bc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.944506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52229628-28d7-4216-b1f9-7ed2a9b3bc2e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.944802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52229628-28d7-4216-b1f9-7ed2a9b3bc2e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.945079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52229628-28d7-4216-b1f9-7ed2a9b3bc2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.959265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52229628-28d7-4216-b1f9-7ed2a9b3bc2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.961455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52229628-28d7-4216-b1f9-7ed2a9b3bc2e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-52229628-28d7-4216-b1f9-7ed2a9b3bc2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:37.976633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:37.977616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:37.979151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:37.979151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:37.979151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:37.979151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 287/576] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:37.979151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52229628-28d7-4216-b1f9-7ed2a9b3bc2e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.984913 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214a039-3a3c-4c7a-b1eb-3e66663adfdd None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.018825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2828313-75b0-41cf-b087-4f29d61406c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.019508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2828313-75b0-41cf-b087-4f29d61406c3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.019795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2828313-75b0-41cf-b087-4f29d61406c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.020071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2828313-75b0-41cf-b087-4f29d61406c3 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.020787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2828313-75b0-41cf-b087-4f29d61406c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:38.021097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2828313-75b0-41cf-b087-4f29d61406c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.022559 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2828313-75b0-41cf-b087-4f29d61406c3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.036461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 290/577] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:38.037208 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2828313-75b0-41cf-b087-4f29d61406c3 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.039949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:38.098073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52229628-28d7-4216-b1f9-7ed2a9b3bc2e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.133022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec2bd4-e2fe-44e4-8490-fcaed78c5e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.133645 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec2bd4-e2fe-44e4-8490-fcaed78c5e61 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.133957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec2bd4-e2fe-44e4-8490-fcaed78c5e61 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.134236 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec2bd4-e2fe-44e4-8490-fcaed78c5e61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.137360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:38.142205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2828313-75b0-41cf-b087-4f29d61406c3 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.146559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec2bd4-e2fe-44e4-8490-fcaed78c5e61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-e2ec2bd4-e2fe-44e4-8490-fcaed78c5e61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:38.151861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:38.153574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 288/578] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:38.154462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec2bd4-e2fe-44e4-8490-fcaed78c5e61 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.185224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869c7b8-f53f-4c47-b364-8e8467d407c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.186066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869c7b8-f53f-4c47-b364-8e8467d407c4 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.186521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869c7b8-f53f-4c47-b364-8e8467d407c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.186953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869c7b8-f53f-4c47-b364-8e8467d407c4 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.187750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869c7b8-f53f-4c47-b364-8e8467d407c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:38.188203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869c7b8-f53f-4c47-b364-8e8467d407c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.189714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869c7b8-f53f-4c47-b364-8e8467d407c4 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.205895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 291/579] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:38.207580 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869c7b8-f53f-4c47-b364-8e8467d407c4 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.259598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec2bd4-e2fe-44e4-8490-fcaed78c5e61 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.272420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:38.294771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6871bf6-15ed-4d40-ba06-13e4cd6254b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.295388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6871bf6-15ed-4d40-ba06-13e4cd6254b7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.295667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6871bf6-15ed-4d40-ba06-13e4cd6254b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.295943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6871bf6-15ed-4d40-ba06-13e4cd6254b7 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.296590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6871bf6-15ed-4d40-ba06-13e4cd6254b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:38.296896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6871bf6-15ed-4d40-ba06-13e4cd6254b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.298300 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6871bf6-15ed-4d40-ba06-13e4cd6254b7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.310808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 289/580] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:38.311444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6871bf6-15ed-4d40-ba06-13e4cd6254b7 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.319547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869c7b8-f53f-4c47-b364-8e8467d407c4 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.360776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6fc71-acc6-4c4a-97e8-0c37cd29ab90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.362040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6fc71-acc6-4c4a-97e8-0c37cd29ab90 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.362318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6fc71-acc6-4c4a-97e8-0c37cd29ab90 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.363540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e6fc71-acc6-4c4a-97e8-0c37cd29ab90 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.372940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6fc71-acc6-4c4a-97e8-0c37cd29ab90 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.373507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:38.375071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e6fc71-acc6-4c4a-97e8-0c37cd29ab90 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-b8e6fc71-acc6-4c4a-97e8-0c37cd29ab90 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:38.387435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:38.388428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:38.389502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:38.389502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:38.389502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:38.389502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:38.389502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:38.389502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:38.389502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:38.389502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:38.389502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:38.389502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:38.389502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 292/581] 10.222.0.55 () {56 vars in 1093 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:38.390146 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6fc71-acc6-4c4a-97e8-0c37cd29ab90 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.427711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6871bf6-15ed-4d40-ba06-13e4cd6254b7 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.466069 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1f0ac-2966-4c54-822d-553c08040009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.466674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1f0ac-2966-4c54-822d-553c08040009 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.466956 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1f0ac-2966-4c54-822d-553c08040009 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.467228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1f0ac-2966-4c54-822d-553c08040009 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.467893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1f0ac-2966-4c54-822d-553c08040009 None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:38.468202 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1f0ac-2966-4c54-822d-553c08040009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.469575 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1f0ac-2966-4c54-822d-553c08040009 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.477514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 290/582] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:38.478336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1f0ac-2966-4c54-822d-553c08040009 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.503691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e6fc71-acc6-4c4a-97e8-0c37cd29ab90 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.532030 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:38.548688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a0410-5fbd-4f06-9a5e-707377f5951b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.549815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a0410-5fbd-4f06-9a5e-707377f5951b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.550338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a0410-5fbd-4f06-9a5e-707377f5951b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.550862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a0410-5fbd-4f06-9a5e-707377f5951b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.560253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a0410-5fbd-4f06-9a5e-707377f5951b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.563326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a0410-5fbd-4f06-9a5e-707377f5951b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-de8a0410-5fbd-4f06-9a5e-707377f5951b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:38.579715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:38.580633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:38.581598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:38.581598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:38.581598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:38.581598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:38.581598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:38.581598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:38.581598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:38.581598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:38.585709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 293/583] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:38.586227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a0410-5fbd-4f06-9a5e-707377f5951b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.595026 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1f0ac-2966-4c54-822d-553c08040009 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.614181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:38.631552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734313ea-ddd4-4900-b024-d8c010813360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.632171 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734313ea-ddd4-4900-b024-d8c010813360 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.632436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734313ea-ddd4-4900-b024-d8c010813360 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.632710 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734313ea-ddd4-4900-b024-d8c010813360 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.640138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734313ea-ddd4-4900-b024-d8c010813360 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.641561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734313ea-ddd4-4900-b024-d8c010813360 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-734313ea-ddd4-4900-b024-d8c010813360 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:38.654160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:38.655127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:38.656059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:38.656059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:38.656059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:38.656059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:38.656059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:38.656059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:38.656059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:38.656059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:38.656059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:39:38.656059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:38.656059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 291/584] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:38.657382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734313ea-ddd4-4900-b024-d8c010813360 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.700722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a0410-5fbd-4f06-9a5e-707377f5951b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.739346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8dc813-9f03-4193-81e1-e8748909c2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.739952 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dc813-9f03-4193-81e1-e8748909c2a2 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.740224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dc813-9f03-4193-81e1-e8748909c2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.740493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8dc813-9f03-4193-81e1-e8748909c2a2 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.741203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8dc813-9f03-4193-81e1-e8748909c2a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:38.741509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8dc813-9f03-4193-81e1-e8748909c2a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.743025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8dc813-9f03-4193-81e1-e8748909c2a2 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.763103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 294/585] 10.222.0.55 () {56 vars in 1133 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:38.764101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734313ea-ddd4-4900-b024-d8c010813360 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.764454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8dc813-9f03-4193-81e1-e8748909c2a2 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.800273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3785dd-556f-47e8-a56d-f89306542e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.801506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3785dd-556f-47e8-a56d-f89306542e42 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.801817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3785dd-556f-47e8-a56d-f89306542e42 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.802198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3785dd-556f-47e8-a56d-f89306542e42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.811402 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:38.813582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3785dd-556f-47e8-a56d-f89306542e42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.815699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3785dd-556f-47e8-a56d-f89306542e42 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-2c3785dd-556f-47e8-a56d-f89306542e42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:38.835023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:38.836000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:38.836908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:38.836908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:38.836908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:38.836908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 292/586] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:38.837587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3785dd-556f-47e8-a56d-f89306542e42 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.876766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:38.877045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8dc813-9f03-4193-81e1-e8748909c2a2 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.916924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66508983-3edf-4e91-8d4f-d97cfff2f910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.917544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66508983-3edf-4e91-8d4f-d97cfff2f910 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.918429 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66508983-3edf-4e91-8d4f-d97cfff2f910 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.918714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66508983-3edf-4e91-8d4f-d97cfff2f910 None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.919384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66508983-3edf-4e91-8d4f-d97cfff2f910 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:38.919690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66508983-3edf-4e91-8d4f-d97cfff2f910 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.921389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66508983-3edf-4e91-8d4f-d97cfff2f910 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.937835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 295/587] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:38.937835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66508983-3edf-4e91-8d4f-d97cfff2f910 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.959231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3785dd-556f-47e8-a56d-f89306542e42 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.994034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7797c-6ca2-493e-98e6-de5a2b91c039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.994650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7797c-6ca2-493e-98e6-de5a2b91c039 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.994927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7797c-6ca2-493e-98e6-de5a2b91c039 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.995205 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7797c-6ca2-493e-98e6-de5a2b91c039 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.995993 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7797c-6ca2-493e-98e6-de5a2b91c039 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:38.996801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7797c-6ca2-493e-98e6-de5a2b91c039 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.998251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7797c-6ca2-493e-98e6-de5a2b91c039 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:39.015081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 293/588] 10.222.0.55 () {56 vars in 1137 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:39.017888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7797c-6ca2-493e-98e6-de5a2b91c039 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:39.048690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:39.053361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66508983-3edf-4e91-8d4f-d97cfff2f910 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:39.091563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc0bdb-4f7a-40af-a2cb-fee8450b2439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:39.092395 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc0bdb-4f7a-40af-a2cb-fee8450b2439 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.092658 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc0bdb-4f7a-40af-a2cb-fee8450b2439 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.092931 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc0bdb-4f7a-40af-a2cb-fee8450b2439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.101323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:39.106934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc0bdb-4f7a-40af-a2cb-fee8450b2439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:39.108621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc0bdb-4f7a-40af-a2cb-fee8450b2439 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:39.113762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7797c-6ca2-493e-98e6-de5a2b91c039 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-e6bc0bdb-4f7a-40af-a2cb-fee8450b2439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:39.124214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:39.125222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:39.126517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:39.126517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:39.126517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:39.126517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 296/589] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:39.126517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc0bdb-4f7a-40af-a2cb-fee8450b2439 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:39.156204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c3b44-2cdf-4067-978d-efc3f88e4017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:39.156738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3b44-2cdf-4067-978d-efc3f88e4017 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.157041 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3b44-2cdf-4067-978d-efc3f88e4017 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.157318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c3b44-2cdf-4067-978d-efc3f88e4017 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.158064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c3b44-2cdf-4067-978d-efc3f88e4017 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:39.158366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c3b44-2cdf-4067-978d-efc3f88e4017 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:39.159731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c3b44-2cdf-4067-978d-efc3f88e4017 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:39.171871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 294/590] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:39.175037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c3b44-2cdf-4067-978d-efc3f88e4017 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:39.244743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc0bdb-4f7a-40af-a2cb-fee8450b2439 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:39.283365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84436be6-3e02-430e-84f6-f51015840ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:39.283931 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84436be6-3e02-430e-84f6-f51015840ab5 None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.286997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84436be6-3e02-430e-84f6-f51015840ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.286997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84436be6-3e02-430e-84f6-f51015840ab5 None admin] PATH_INFO: `/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/4e27f5b81c704dbebbd6a83b2713e237/roles/7377909b229e4449a4cbe1a22f8bf1f8` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.293505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c3b44-2cdf-4067-978d-efc3f88e4017 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:39.300342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84436be6-3e02-430e-84f6-f51015840ab5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe9c4e7bc14e70b57bf834cfff365f, user_id=4e27f5b81c704dbebbd6a83b2713e237, role_id=7377909b229e4449a4cbe1a22f8bf1f8)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:39.301945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84436be6-3e02-430e-84f6-f51015840ab5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:39.315664 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 297/591] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:39 2022] PUT /identity/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/4e27f5b81c704dbebbd6a83b2713e237/roles/7377909b229e4449a4cbe1a22f8bf1f8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:39.316470 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84436be6-3e02-430e-84f6-f51015840ab5 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:39.319443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:39.344715 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9176de-dfda-4423-819d-4255caed73ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:39.345363 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9176de-dfda-4423-819d-4255caed73ae None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.345642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9176de-dfda-4423-819d-4255caed73ae None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.345950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9176de-dfda-4423-819d-4255caed73ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.360020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9176de-dfda-4423-819d-4255caed73ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:39.362102 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9176de-dfda-4423-819d-4255caed73ae None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:39.373355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-fa9176de-dfda-4423-819d-4255caed73ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:39.380481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:39.381724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:39.382849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:39.382849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:39.382849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:39.385369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 295/592] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:39.386445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9176de-dfda-4423-819d-4255caed73ae None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:39.443501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84436be6-3e02-430e-84f6-f51015840ab5 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:39.483619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3cefb0-1986-4d52-a21b-1a8a3a8e1eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:39.484390 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cefb0-1986-4d52-a21b-1a8a3a8e1eff None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.484765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cefb0-1986-4d52-a21b-1a8a3a8e1eff None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.485163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3cefb0-1986-4d52-a21b-1a8a3a8e1eff None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.485950 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3cefb0-1986-4d52-a21b-1a8a3a8e1eff None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:39.486255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3cefb0-1986-4d52-a21b-1a8a3a8e1eff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:39.488274 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3cefb0-1986-4d52-a21b-1a8a3a8e1eff None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:39.504653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 298/593] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:39.507261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0510914-0e31-42a6-b4b5-3173ebc23a3d None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.507261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0510914-0e31-42a6-b4b5-3173ebc23a3d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.507261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0510914-0e31-42a6-b4b5-3173ebc23a3d None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.507931 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 299/594] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:39.508049 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9176de-dfda-4423-819d-4255caed73ae None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:39.508474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0510914-0e31-42a6-b4b5-3173ebc23a3d None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:39.545868 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0476c-77ed-4009-8640-515f29e66111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:39.546587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0476c-77ed-4009-8640-515f29e66111 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.546960 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0476c-77ed-4009-8640-515f29e66111 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.547308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0476c-77ed-4009-8640-515f29e66111 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.554973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0476c-77ed-4009-8640-515f29e66111 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:39.556409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0476c-77ed-4009-8640-515f29e66111 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-00e0476c-77ed-4009-8640-515f29e66111 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:39.568403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:39.569870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:39.570801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:39.570801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:39.570801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:39.570801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:39.570801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:39.570801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:39.570801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:39.570801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:39.570801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:39:39.570801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:39.570801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 296/595] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:39.570801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0476c-77ed-4009-8640-515f29e66111 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:39.581203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:39.622517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0510914-0e31-42a6-b4b5-3173ebc23a3d None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:39.637942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:39.665038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73febd31-17b0-4737-9fe9-c0df53309a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:39.666130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73febd31-17b0-4737-9fe9-c0df53309a63 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.666702 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73febd31-17b0-4737-9fe9-c0df53309a63 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.666979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73febd31-17b0-4737-9fe9-c0df53309a63 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.668265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73febd31-17b0-4737-9fe9-c0df53309a63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:39.668569 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73febd31-17b0-4737-9fe9-c0df53309a63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:39.670828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73febd31-17b0-4737-9fe9-c0df53309a63 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:39.687768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 300/596] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:39.687768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e992b5-64e0-4dcf-bca5-74256bf556ad None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.687768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e992b5-64e0-4dcf-bca5-74256bf556ad None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.687768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e992b5-64e0-4dcf-bca5-74256bf556ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.692522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0476c-77ed-4009-8640-515f29e66111 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:39.704967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-a2e992b5-64e0-4dcf-bca5-74256bf556ad None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:39.706113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a2e992b5-64e0-4dcf-bca5-74256bf556ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:39.728061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654db4f-ae5a-4c84-b5fe-e3186c317b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:39.728696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654db4f-ae5a-4c84-b5fe-e3186c317b62 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.728974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654db4f-ae5a-4c84-b5fe-e3186c317b62 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.729247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654db4f-ae5a-4c84-b5fe-e3186c317b62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.736634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654db4f-ae5a-4c84-b5fe-e3186c317b62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:39.738088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654db4f-ae5a-4c84-b5fe-e3186c317b62 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-d654db4f-ae5a-4c84-b5fe-e3186c317b62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:39.749855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:39.750974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:39.756735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:39.756735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:39.756735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:39.756735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:39.756735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:39.756735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:39.756735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:39.756735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:39.756735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 297/597] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:39.756735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654db4f-ae5a-4c84-b5fe-e3186c317b62 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:39.787420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 301/598] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:39.788272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e992b5-64e0-4dcf-bca5-74256bf556ad None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:39.862757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:39.862988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654db4f-ae5a-4c84-b5fe-e3186c317b62 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:39.888729 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:39.919037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08e71f-e6c6-4e29-af53-d81ac9b6967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:39.919562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08e71f-e6c6-4e29-af53-d81ac9b6967f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.919846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08e71f-e6c6-4e29-af53-d81ac9b6967f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.920120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08e71f-e6c6-4e29-af53-d81ac9b6967f None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.920854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08e71f-e6c6-4e29-af53-d81ac9b6967f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:39.921161 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08e71f-e6c6-4e29-af53-d81ac9b6967f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:39.922614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08e71f-e6c6-4e29-af53-d81ac9b6967f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:39.924104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e992b5-64e0-4dcf-bca5-74256bf556ad None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:39.945336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 298/599] 10.222.0.55 () {56 vars in 1139 bytes} [Thu Sep 8 13:39:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:39.946345 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08e71f-e6c6-4e29-af53-d81ac9b6967f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:39.961372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5363f-ac20-4ce5-b858-d6f57bbfbb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:39.962314 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5363f-ac20-4ce5-b858-d6f57bbfbb48 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.962761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5363f-ac20-4ce5-b858-d6f57bbfbb48 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.963194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5363f-ac20-4ce5-b858-d6f57bbfbb48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.971402 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5363f-ac20-4ce5-b858-d6f57bbfbb48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:39.973196 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5363f-ac20-4ce5-b858-d6f57bbfbb48 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-d9f5363f-ac20-4ce5-b858-d6f57bbfbb48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:39.986141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:39.987249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:39.988287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:39.988287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:39.988287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:39.988287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:39.988287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:39.988287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:39.988287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:39.988287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:39.995009 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 302/600] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:39.996171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5363f-ac20-4ce5-b858-d6f57bbfbb48 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.050355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08e71f-e6c6-4e29-af53-d81ac9b6967f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.085633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74df83-5a7c-4f20-861a-c9b6bd041c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.086251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74df83-5a7c-4f20-861a-c9b6bd041c5f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.086532 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74df83-5a7c-4f20-861a-c9b6bd041c5f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.086804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74df83-5a7c-4f20-861a-c9b6bd041c5f None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.087496 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74df83-5a7c-4f20-861a-c9b6bd041c5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:40.087802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74df83-5a7c-4f20-861a-c9b6bd041c5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.090134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74df83-5a7c-4f20-861a-c9b6bd041c5f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.101187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 299/601] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:40.101187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74df83-5a7c-4f20-861a-c9b6bd041c5f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.115626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5363f-ac20-4ce5-b858-d6f57bbfbb48 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.129502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:40.142068 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:40.156675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f200c-4b39-4c83-b65d-9196c361a5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.157594 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f200c-4b39-4c83-b65d-9196c361a5b1 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.158089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f200c-4b39-4c83-b65d-9196c361a5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.158780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f200c-4b39-4c83-b65d-9196c361a5b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.172431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f200c-4b39-4c83-b65d-9196c361a5b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.175281 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f200c-4b39-4c83-b65d-9196c361a5b1 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-9f1f200c-4b39-4c83-b65d-9196c361a5b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:40.191407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:40.192443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:40.193372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:40.193372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:40.193372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:40.193372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 303/602] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:40.193372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f200c-4b39-4c83-b65d-9196c361a5b1 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.210674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74df83-5a7c-4f20-861a-c9b6bd041c5f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.248995 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae91068-81d1-4e27-8cc9-b7b93d1c8bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.249831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae91068-81d1-4e27-8cc9-b7b93d1c8bdc None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.250118 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae91068-81d1-4e27-8cc9-b7b93d1c8bdc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.250393 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae91068-81d1-4e27-8cc9-b7b93d1c8bdc None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.251090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae91068-81d1-4e27-8cc9-b7b93d1c8bdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:40.251398 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae91068-81d1-4e27-8cc9-b7b93d1c8bdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.253258 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae91068-81d1-4e27-8cc9-b7b93d1c8bdc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.268530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 300/603] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:40.269343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae91068-81d1-4e27-8cc9-b7b93d1c8bdc None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.310493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f200c-4b39-4c83-b65d-9196c361a5b1 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.350305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf4a40-0d1e-4100-ab5c-183fea168d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.351187 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf4a40-0d1e-4100-ab5c-183fea168d95 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.351631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf4a40-0d1e-4100-ab5c-183fea168d95 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.352068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf4a40-0d1e-4100-ab5c-183fea168d95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.363208 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf4a40-0d1e-4100-ab5c-183fea168d95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:40.367697 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-0ddf4a40-0d1e-4100-ab5c-183fea168d95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:40.368649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:40.369893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 304/604] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:40.370742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf4a40-0d1e-4100-ab5c-183fea168d95 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.378779 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae91068-81d1-4e27-8cc9-b7b93d1c8bdc None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.401865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:40.418139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806d2d4-f30c-431a-a7c6-5c5d69a4d064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.418618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d2d4-f30c-431a-a7c6-5c5d69a4d064 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.418964 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d2d4-f30c-431a-a7c6-5c5d69a4d064 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.419307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806d2d4-f30c-431a-a7c6-5c5d69a4d064 None admin] PATH_INFO: `/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/6ffa73f81539472391983c51447af1b6/roles/7377909b229e4449a4cbe1a22f8bf1f8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.434710 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806d2d4-f30c-431a-a7c6-5c5d69a4d064 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe9c4e7bc14e70b57bf834cfff365f, user_id=6ffa73f81539472391983c51447af1b6, role_id=7377909b229e4449a4cbe1a22f8bf1f8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.436252 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806d2d4-f30c-431a-a7c6-5c5d69a4d064 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.451263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 301/605] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:40 2022] PUT /identity/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/6ffa73f81539472391983c51447af1b6/roles/7377909b229e4449a4cbe1a22f8bf1f8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:40.451953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806d2d4-f30c-431a-a7c6-5c5d69a4d064 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.515234 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf4a40-0d1e-4100-ab5c-183fea168d95 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.566063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806d2d4-f30c-431a-a7c6-5c5d69a4d064 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.568170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c549ae-8056-45c2-b2cf-eef6a412af91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.570226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c549ae-8056-45c2-b2cf-eef6a412af91 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.570511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c549ae-8056-45c2-b2cf-eef6a412af91 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.574578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c549ae-8056-45c2-b2cf-eef6a412af91 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.575583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c549ae-8056-45c2-b2cf-eef6a412af91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:40.576826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c549ae-8056-45c2-b2cf-eef6a412af91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.579079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c549ae-8056-45c2-b2cf-eef6a412af91 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.599754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 305/606] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:40.601009 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c549ae-8056-45c2-b2cf-eef6a412af91 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.612344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70057d6a-29ec-4a08-95df-55b5968cb25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.613025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70057d6a-29ec-4a08-95df-55b5968cb25e None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.613306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70057d6a-29ec-4a08-95df-55b5968cb25e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.613584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70057d6a-29ec-4a08-95df-55b5968cb25e None admin] PATH_INFO: `/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/ff93243f0c4c48d1924ea04a87a0e7c1/roles/7377909b229e4449a4cbe1a22f8bf1f8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.631414 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70057d6a-29ec-4a08-95df-55b5968cb25e None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe9c4e7bc14e70b57bf834cfff365f, user_id=ff93243f0c4c48d1924ea04a87a0e7c1, role_id=7377909b229e4449a4cbe1a22f8bf1f8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.632873 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70057d6a-29ec-4a08-95df-55b5968cb25e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.645648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:40.654885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 302/607] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:40 2022] PUT /identity/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/ff93243f0c4c48d1924ea04a87a0e7c1/roles/7377909b229e4449a4cbe1a22f8bf1f8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:40.656002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70057d6a-29ec-4a08-95df-55b5968cb25e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.781053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:40.816099 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c549ae-8056-45c2-b2cf-eef6a412af91 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.836027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70057d6a-29ec-4a08-95df-55b5968cb25e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.888164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f4b49-d82d-4ab5-999d-19b29f95af06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.888784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f4b49-d82d-4ab5-999d-19b29f95af06 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.889068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f4b49-d82d-4ab5-999d-19b29f95af06 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.889340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965f4b49-d82d-4ab5-999d-19b29f95af06 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.890073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f4b49-d82d-4ab5-999d-19b29f95af06 None admin] RBAC: Adding query filter params (name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:40.890378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f4b49-d82d-4ab5-999d-19b29f95af06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.891752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965f4b49-d82d-4ab5-999d-19b29f95af06 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.892238 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca5780-e5c9-418d-86b5-a6e56a63843a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.893628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca5780-e5c9-418d-86b5-a6e56a63843a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.894526 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca5780-e5c9-418d-86b5-a6e56a63843a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.894972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca5780-e5c9-418d-86b5-a6e56a63843a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.905674 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 306/608] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:40.912642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca5780-e5c9-418d-86b5-a6e56a63843a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.912642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca5780-e5c9-418d-86b5-a6e56a63843a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.914367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f4b49-d82d-4ab5-999d-19b29f95af06 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-68ca5780-e5c9-418d-86b5-a6e56a63843a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:40.926529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:40.927648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:40.928814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:40.928814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:40.928814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:40.928814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:40.928814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:40.928814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:40.928814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:40.928814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:40.928814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 303/609] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:40.934098 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca5780-e5c9-418d-86b5-a6e56a63843a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.996124 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:41.045861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965f4b49-d82d-4ab5-999d-19b29f95af06 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:41.046058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca5780-e5c9-418d-86b5-a6e56a63843a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:41.106935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab458863-154b-4e44-8b2a-adcd942a049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:41.108108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab458863-154b-4e44-8b2a-adcd942a049b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.108872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab458863-154b-4e44-8b2a-adcd942a049b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.109524 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab458863-154b-4e44-8b2a-adcd942a049b None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.110815 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab458863-154b-4e44-8b2a-adcd942a049b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:41.111600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47266e2-9ceb-414c-abfe-137487066b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:41.112301 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47266e2-9ceb-414c-abfe-137487066b9f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.112681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab458863-154b-4e44-8b2a-adcd942a049b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:41.113024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47266e2-9ceb-414c-abfe-137487066b9f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.114371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47266e2-9ceb-414c-abfe-137487066b9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.115475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab458863-154b-4e44-8b2a-adcd942a049b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:41.135479 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 304/610] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:41.136868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47266e2-9ceb-414c-abfe-137487066b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:41.138438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47266e2-9ceb-414c-abfe-137487066b9f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:41.144630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab458863-154b-4e44-8b2a-adcd942a049b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:41.160972 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-b47266e2-9ceb-414c-abfe-137487066b9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:41.175124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:41.176112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:41.177403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:41.177403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:41.177403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:41.177403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 307/611] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:41.181594 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47266e2-9ceb-414c-abfe-137487066b9f None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:41.337112 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab458863-154b-4e44-8b2a-adcd942a049b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:41.368119 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:41.374198 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47266e2-9ceb-414c-abfe-137487066b9f None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:41.404711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941a8d7-2e77-4b07-8508-26f130695b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:41.405376 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941a8d7-2e77-4b07-8508-26f130695b1a None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.405656 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941a8d7-2e77-4b07-8508-26f130695b1a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.405957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941a8d7-2e77-4b07-8508-26f130695b1a None admin] PATH_INFO: `/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/531ebc79bf424e9b8762b2c7ec7fe8d0/roles/7377909b229e4449a4cbe1a22f8bf1f8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.435741 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941a8d7-2e77-4b07-8508-26f130695b1a None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe9c4e7bc14e70b57bf834cfff365f, user_id=531ebc79bf424e9b8762b2c7ec7fe8d0, role_id=7377909b229e4449a4cbe1a22f8bf1f8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:41.437874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941a8d7-2e77-4b07-8508-26f130695b1a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:41.444897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d6a88-e90c-4a42-b4e0-24a2084aa0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:41.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d6a88-e90c-4a42-b4e0-24a2084aa0d2 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.445892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d6a88-e90c-4a42-b4e0-24a2084aa0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.446328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d6a88-e90c-4a42-b4e0-24a2084aa0d2 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.447065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d6a88-e90c-4a42-b4e0-24a2084aa0d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:41.447386 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d6a88-e90c-4a42-b4e0-24a2084aa0d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:41.449646 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d6a88-e90c-4a42-b4e0-24a2084aa0d2 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:41.463485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 305/612] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:41 2022] PUT /identity/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/531ebc79bf424e9b8762b2c7ec7fe8d0/roles/7377909b229e4449a4cbe1a22f8bf1f8 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:41.481022 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 308/613] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:41.486378 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941a8d7-2e77-4b07-8508-26f130695b1a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:41.548583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b63b02-95b8-4dad-b988-9274627ec1c0 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.548893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b63b02-95b8-4dad-b988-9274627ec1c0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.549135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b63b02-95b8-4dad-b988-9274627ec1c0 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.550975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 309/614] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:41.559545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a613442-0742-49d1-b058-755ba724b78c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.560002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a613442-0742-49d1-b058-755ba724b78c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.560372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a613442-0742-49d1-b058-755ba724b78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.593215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1a613442-0742-49d1-b058-755ba724b78c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:41.595587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1a613442-0742-49d1-b058-755ba724b78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:41.667158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941a8d7-2e77-4b07-8508-26f130695b1a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:41.710430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32e138-3bf0-499d-807d-8f325bbf9c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:41.711073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32e138-3bf0-499d-807d-8f325bbf9c18 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.711353 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32e138-3bf0-499d-807d-8f325bbf9c18 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.711628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32e138-3bf0-499d-807d-8f325bbf9c18 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.720561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32e138-3bf0-499d-807d-8f325bbf9c18 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:41.722041 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32e138-3bf0-499d-807d-8f325bbf9c18 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-dc32e138-3bf0-499d-807d-8f325bbf9c18 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:41.735221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:41.736201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:41.737164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:41.737164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:41.737164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:41.737164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:41.737164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:41.737164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:41.737164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:41.737164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:41.737164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:41.737164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:41.737164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 306/615] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:41.741093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:41.750888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32e138-3bf0-499d-807d-8f325bbf9c18 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:41.769552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 310/616] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:39:41.789642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:41.878621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32e138-3bf0-499d-807d-8f325bbf9c18 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:41.926320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4c1ae-2e2b-4891-bb43-acf32e1440cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:41.926921 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4c1ae-2e2b-4891-bb43-acf32e1440cd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.927191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4c1ae-2e2b-4891-bb43-acf32e1440cd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.927461 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a4c1ae-2e2b-4891-bb43-acf32e1440cd None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.928187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4c1ae-2e2b-4891-bb43-acf32e1440cd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:41.928488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4c1ae-2e2b-4891-bb43-acf32e1440cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:41.929943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a4c1ae-2e2b-4891-bb43-acf32e1440cd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:41.957188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 307/617] 10.222.0.55 () {56 vars in 1135 bytes} [Thu Sep 8 13:39:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:41.962634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a613442-0742-49d1-b058-755ba724b78c None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.090838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a613442-0742-49d1-b058-755ba724b78c None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.125111 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19bd2f-17df-416c-8f45-ad766b0c13df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.125756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19bd2f-17df-416c-8f45-ad766b0c13df None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.126040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19bd2f-17df-416c-8f45-ad766b0c13df None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.126308 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19bd2f-17df-416c-8f45-ad766b0c13df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.137013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19bd2f-17df-416c-8f45-ad766b0c13df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:42.138412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19bd2f-17df-416c-8f45-ad766b0c13df None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-8e19bd2f-17df-416c-8f45-ad766b0c13df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:42.153105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:42.154101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:42.155008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:42.155008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:42.155008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:42.155008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 311/618] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:42.162135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4c1ae-2e2b-4891-bb43-acf32e1440cd None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.182453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19bd2f-17df-416c-8f45-ad766b0c13df None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.236021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:42.299238 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19bd2f-17df-416c-8f45-ad766b0c13df None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.337044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766990c0-b461-4d21-a30f-1aa25e236776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.337677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766990c0-b461-4d21-a30f-1aa25e236776 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.337978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766990c0-b461-4d21-a30f-1aa25e236776 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.338253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766990c0-b461-4d21-a30f-1aa25e236776 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.345329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:42.357401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766990c0-b461-4d21-a30f-1aa25e236776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-766990c0-b461-4d21-a30f-1aa25e236776 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:42.362673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:42.363757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:42.364596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:42.364596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:42.364596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:42.364596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:42.364596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:42.364596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:42.364596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:42.364596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:42.364596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:42.364596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:42.364596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:42.364596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:42.365279 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 312/619] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:42.372703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766990c0-b461-4d21-a30f-1aa25e236776 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.383662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a4c1ae-2e2b-4891-bb43-acf32e1440cd None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.461321 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607b2a0-67d2-46cc-883a-fcdcb6befa11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.461874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607b2a0-67d2-46cc-883a-fcdcb6befa11 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.462158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607b2a0-67d2-46cc-883a-fcdcb6befa11 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.462433 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607b2a0-67d2-46cc-883a-fcdcb6befa11 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.463081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607b2a0-67d2-46cc-883a-fcdcb6befa11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:42.463382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607b2a0-67d2-46cc-883a-fcdcb6befa11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:42.464748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6607b2a0-67d2-46cc-883a-fcdcb6befa11 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:42.490622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 308/620] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:42.493429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607b2a0-67d2-46cc-883a-fcdcb6befa11 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.551396 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766990c0-b461-4d21-a30f-1aa25e236776 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.555832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:42.597662 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e798007-7e1e-48c4-b690-c50419f5586f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.598678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e798007-7e1e-48c4-b690-c50419f5586f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.599160 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e798007-7e1e-48c4-b690-c50419f5586f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.599817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e798007-7e1e-48c4-b690-c50419f5586f None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.601721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e798007-7e1e-48c4-b690-c50419f5586f None admin] RBAC: Adding query filter params (name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:42.602661 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e798007-7e1e-48c4-b690-c50419f5586f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:42.604578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e798007-7e1e-48c4-b690-c50419f5586f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:42.621860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 313/621] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:42.626230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e798007-7e1e-48c4-b690-c50419f5586f None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.680188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607b2a0-67d2-46cc-883a-fcdcb6befa11 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.721325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1267ac-3afa-4f6a-8892-a5dddcb5e836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.721933 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1267ac-3afa-4f6a-8892-a5dddcb5e836 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.722217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1267ac-3afa-4f6a-8892-a5dddcb5e836 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.722488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1267ac-3afa-4f6a-8892-a5dddcb5e836 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.730932 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1267ac-3afa-4f6a-8892-a5dddcb5e836 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:42.733131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1267ac-3afa-4f6a-8892-a5dddcb5e836 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-6d1267ac-3afa-4f6a-8892-a5dddcb5e836 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:42.747980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:42.748982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:42.749922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:42.749922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:42.749922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:42.749922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:42.749922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:42.749922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:42.749922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:42.749922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:42.749922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 309/622] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:42.755216 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1267ac-3afa-4f6a-8892-a5dddcb5e836 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.768119 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:42.803865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e798007-7e1e-48c4-b690-c50419f5586f None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.868173 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1267ac-3afa-4f6a-8892-a5dddcb5e836 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.874997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efa7de-e894-421e-9e28-3577e7bd18d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.875935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efa7de-e894-421e-9e28-3577e7bd18d6 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.876220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efa7de-e894-421e-9e28-3577e7bd18d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.876494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11efa7de-e894-421e-9e28-3577e7bd18d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.907006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efa7de-e894-421e-9e28-3577e7bd18d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:42.908834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11efa7de-e894-421e-9e28-3577e7bd18d6 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:42.921832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cce95f-5236-4f79-b9ea-0af818dfd3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.922977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cce95f-5236-4f79-b9ea-0af818dfd3ad None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.923530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cce95f-5236-4f79-b9ea-0af818dfd3ad None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.924028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cce95f-5236-4f79-b9ea-0af818dfd3ad None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.924981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cce95f-5236-4f79-b9ea-0af818dfd3ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:42.925574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cce95f-5236-4f79-b9ea-0af818dfd3ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:42.927414 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cce95f-5236-4f79-b9ea-0af818dfd3ad None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:42.944276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 310/623] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-11efa7de-e894-421e-9e28-3577e7bd18d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:42.950973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:42.952003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:42.952932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:42.952932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:42.952932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:42.952932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 314/624] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:42.953811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cce95f-5236-4f79-b9ea-0af818dfd3ad None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.962676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efa7de-e894-421e-9e28-3577e7bd18d6 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:43.003641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:43.037077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:43.086531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cce95f-5236-4f79-b9ea-0af818dfd3ad None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:43.128102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11efa7de-e894-421e-9e28-3577e7bd18d6 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:43.135245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f858c1f3-6b5d-4346-9477-a4bc82c610f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:43.135905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858c1f3-6b5d-4346-9477-a4bc82c610f9 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.136200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858c1f3-6b5d-4346-9477-a4bc82c610f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.137007 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f858c1f3-6b5d-4346-9477-a4bc82c610f9 None admin] PATH_INFO: `/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/4e5d73e8daa84975bc21e41e3cbef9cb/roles/7377909b229e4449a4cbe1a22f8bf1f8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.151856 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f858c1f3-6b5d-4346-9477-a4bc82c610f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe9c4e7bc14e70b57bf834cfff365f, user_id=4e5d73e8daa84975bc21e41e3cbef9cb, role_id=7377909b229e4449a4cbe1a22f8bf1f8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:43.153288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f858c1f3-6b5d-4346-9477-a4bc82c610f9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:43.175046 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 311/625] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:42 2022] PUT /identity/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/4e5d73e8daa84975bc21e41e3cbef9cb/roles/7377909b229e4449a4cbe1a22f8bf1f8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:39:43.175046 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da86b3-650a-4c2c-80cf-205bb92e5995 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.175046 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da86b3-650a-4c2c-80cf-205bb92e5995 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.175046 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da86b3-650a-4c2c-80cf-205bb92e5995 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.179250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 312/626] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:43.179772 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cefa6f-3a49-4f2c-9dcc-0928c863e455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:43.180680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cefa6f-3a49-4f2c-9dcc-0928c863e455 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.181025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e8d61-dbb6-41ce-9553-7fe17a1555e3 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.181368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e8d61-dbb6-41ce-9553-7fe17a1555e3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.181623 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cefa6f-3a49-4f2c-9dcc-0928c863e455 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.181764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e8d61-dbb6-41ce-9553-7fe17a1555e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.189773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cefa6f-3a49-4f2c-9dcc-0928c863e455 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.189773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cefa6f-3a49-4f2c-9dcc-0928c863e455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:43.189773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cefa6f-3a49-4f2c-9dcc-0928c863e455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:43.189773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cefa6f-3a49-4f2c-9dcc-0928c863e455 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:43.225278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-378e8d61-dbb6-41ce-9553-7fe17a1555e3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:43.226141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 315/627] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:43.227362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-378e8d61-dbb6-41ce-9553-7fe17a1555e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:43.231923 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cefa6f-3a49-4f2c-9dcc-0928c863e455 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:43.361945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 313/628] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:39:43.363431 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd4e92-6c79-4842-acac-95e7446b64d2 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.363746 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd4e92-6c79-4842-acac-95e7446b64d2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.364023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd4e92-6c79-4842-acac-95e7446b64d2 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.370044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:43.373357 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 314/629] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:43 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:43.374917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd7769-751f-49d8-b5bd-9ca93771268a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.375203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd7769-751f-49d8-b5bd-9ca93771268a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.375477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd7769-751f-49d8-b5bd-9ca93771268a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.408601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cefa6f-3a49-4f2c-9dcc-0928c863e455 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:43.409158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c2cd7769-751f-49d8-b5bd-9ca93771268a None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:43.410849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c2cd7769-751f-49d8-b5bd-9ca93771268a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:43.458090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b172a-afe2-445b-adae-688569c1a600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:43.459248 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b172a-afe2-445b-adae-688569c1a600 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.459734 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b172a-afe2-445b-adae-688569c1a600 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.460206 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b172a-afe2-445b-adae-688569c1a600 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.465595 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:43.472751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b172a-afe2-445b-adae-688569c1a600 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:43.475814 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b172a-afe2-445b-adae-688569c1a600 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-d30b172a-afe2-445b-adae-688569c1a600 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:43.492632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:43.493627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:43.494630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:43.494630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:43.494630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:43.494630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:43.494630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:43.494630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:43.494630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:43.494630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:43.494630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:39:43.494630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:43.494630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 316/630] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:43 2022] GET /identity/v3/users/glance => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:43.499581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b172a-afe2-445b-adae-688569c1a600 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:43.504424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 315/631] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:43.602039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b172a-afe2-445b-adae-688569c1a600 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:43.637512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c645a8-da7c-45f7-9cc4-f13fc09c4889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:43.638160 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c645a8-da7c-45f7-9cc4-f13fc09c4889 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.638439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c645a8-da7c-45f7-9cc4-f13fc09c4889 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.638712 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c645a8-da7c-45f7-9cc4-f13fc09c4889 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.639449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c645a8-da7c-45f7-9cc4-f13fc09c4889 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:43.639999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c645a8-da7c-45f7-9cc4-f13fc09c4889 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:43.641156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd7769-751f-49d8-b5bd-9ca93771268a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:43.641290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c645a8-da7c-45f7-9cc4-f13fc09c4889 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:43.650075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:43.671533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 317/632] 10.222.0.55 () {56 vars in 1137 bytes} [Thu Sep 8 13:39:43 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:43.677775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c645a8-da7c-45f7-9cc4-f13fc09c4889 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:43.846773 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd7769-751f-49d8-b5bd-9ca93771268a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:43.846972 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c645a8-da7c-45f7-9cc4-f13fc09c4889 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:43.883325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1009b-8b4b-438f-b25c-558cb5df1f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:43.883937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1009b-8b4b-438f-b25c-558cb5df1f57 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.884219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1009b-8b4b-438f-b25c-558cb5df1f57 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.884503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1009b-8b4b-438f-b25c-558cb5df1f57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.888420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339010a-d42d-4c0f-b462-43f2d2323fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:43.889373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339010a-d42d-4c0f-b462-43f2d2323fa2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.889866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339010a-d42d-4c0f-b462-43f2d2323fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.890340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339010a-d42d-4c0f-b462-43f2d2323fa2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.903731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339010a-d42d-4c0f-b462-43f2d2323fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-8339010a-d42d-4c0f-b462-43f2d2323fa2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:43.908927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:43.909900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:43.911126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:43.911126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:43.911126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:43.911126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:43.911126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:43.911126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:43.911126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:43.911126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:43.911126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:43.911126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:43.911126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:43.911126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:43.911126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 316/633] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:43.911820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339010a-d42d-4c0f-b462-43f2d2323fa2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:43.947971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:43.950435 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:43.962751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1009b-8b4b-438f-b25c-558cb5df1f57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:43.965416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1009b-8b4b-438f-b25c-558cb5df1f57 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.030451 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339010a-d42d-4c0f-b462-43f2d2323fa2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.065938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55f225-3135-4c5e-b721-c793bfbbfde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.066594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55f225-3135-4c5e-b721-c793bfbbfde9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.066930 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55f225-3135-4c5e-b721-c793bfbbfde9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.067213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55f225-3135-4c5e-b721-c793bfbbfde9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.080988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b55f225-3135-4c5e-b721-c793bfbbfde9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-2b55f225-3135-4c5e-b721-c793bfbbfde9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:44.086259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:44.087501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:44.088534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:44.088534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:44.088534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:44.088534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:44.088534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:44.088534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:44.088534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:44.088534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:44.088534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:44.088534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:44.088534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:44.088534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:44.089263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 317/634] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:44.093878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55f225-3135-4c5e-b721-c793bfbbfde9 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.100611 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 318/635] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:44.101610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1009b-8b4b-438f-b25c-558cb5df1f57 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.197073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.204344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55f225-3135-4c5e-b721-c793bfbbfde9 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.208707 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.218434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1009b-8b4b-438f-b25c-558cb5df1f57 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.243236 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2013a7e-2b23-4188-90d7-7ea2faafa3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.243846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2013a7e-2b23-4188-90d7-7ea2faafa3a3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.244123 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2013a7e-2b23-4188-90d7-7ea2faafa3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.244397 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2013a7e-2b23-4188-90d7-7ea2faafa3a3 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.245061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2013a7e-2b23-4188-90d7-7ea2faafa3a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:44.245363 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2013a7e-2b23-4188-90d7-7ea2faafa3a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.246792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2013a7e-2b23-4188-90d7-7ea2faafa3a3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.254825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb97fdb-6481-4444-8c2b-6abb8a3cd143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.254997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 318/636] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:44.255647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2013a7e-2b23-4188-90d7-7ea2faafa3a3 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.256005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb97fdb-6481-4444-8c2b-6abb8a3cd143 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.256290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb97fdb-6481-4444-8c2b-6abb8a3cd143 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.256589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb97fdb-6481-4444-8c2b-6abb8a3cd143 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.257259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb97fdb-6481-4444-8c2b-6abb8a3cd143 None admin] RBAC: Adding query filter params (name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:44.257569 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb97fdb-6481-4444-8c2b-6abb8a3cd143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.258951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb97fdb-6481-4444-8c2b-6abb8a3cd143 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.268337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 319/637] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:44.269622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec7c2f-5637-4b15-a373-e69809d7e826 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.269940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec7c2f-5637-4b15-a373-e69809d7e826 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.270216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec7c2f-5637-4b15-a373-e69809d7e826 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.271557 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 320/638] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:44.272096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec7c2f-5637-4b15-a373-e69809d7e826 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.371349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2013a7e-2b23-4188-90d7-7ea2faafa3a3 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.384271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec7c2f-5637-4b15-a373-e69809d7e826 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.406217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee349555-c542-43b5-ac6c-ccc46d468520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.406661 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee349555-c542-43b5-ac6c-ccc46d468520 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.406941 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee349555-c542-43b5-ac6c-ccc46d468520 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.407214 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee349555-c542-43b5-ac6c-ccc46d468520 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.417797 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee349555-c542-43b5-ac6c-ccc46d468520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.419339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee349555-c542-43b5-ac6c-ccc46d468520 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.421981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8cc5a-f6d3-4bb9-8cd7-8375a620d3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.422607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8cc5a-f6d3-4bb9-8cd7-8375a620d3d7 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.422888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8cc5a-f6d3-4bb9-8cd7-8375a620d3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.423217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8cc5a-f6d3-4bb9-8cd7-8375a620d3d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-ee349555-c542-43b5-ac6c-ccc46d468520 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:44.434242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:44.435292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:44.436178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:44.436178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:44.436178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:44.436178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 319/639] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:44.436875 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49b043-d0eb-40ef-a3f7-fd9e64347ec6 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.436875 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49b043-d0eb-40ef-a3f7-fd9e64347ec6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.437014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49b043-d0eb-40ef-a3f7-fd9e64347ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.440523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8cc5a-f6d3-4bb9-8cd7-8375a620d3d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.441955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8cc5a-f6d3-4bb9-8cd7-8375a620d3d7 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.456256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9b49b043-d0eb-40ef-a3f7-fd9e64347ec6 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:44.456642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9b49b043-d0eb-40ef-a3f7-fd9e64347ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:44.460298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-b0d8cc5a-f6d3-4bb9-8cd7-8375a620d3d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:44.463143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:44.464091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:44.470632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:44.470632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:44.470632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:44.470632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 321/640] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:44.470632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8cc5a-f6d3-4bb9-8cd7-8375a620d3d7 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.480154 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.541784 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 320/641] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:44.541784 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49b043-d0eb-40ef-a3f7-fd9e64347ec6 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.603508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8cc5a-f6d3-4bb9-8cd7-8375a620d3d7 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.637533 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49b043-d0eb-40ef-a3f7-fd9e64347ec6 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.651686 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05f522-840a-49ec-b8db-85024503c99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.653312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f522-840a-49ec-b8db-85024503c99f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.653866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f522-840a-49ec-b8db-85024503c99f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.654293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e05f522-840a-49ec-b8db-85024503c99f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.671117 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05f522-840a-49ec-b8db-85024503c99f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.673076 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e52a9-ab9e-4902-9045-3bc526704bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.673380 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e05f522-840a-49ec-b8db-85024503c99f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.673922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e52a9-ab9e-4902-9045-3bc526704bbe None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.674298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e52a9-ab9e-4902-9045-3bc526704bbe None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.674667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e52a9-ab9e-4902-9045-3bc526704bbe None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.675403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e52a9-ab9e-4902-9045-3bc526704bbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:44.675804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e52a9-ab9e-4902-9045-3bc526704bbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.677251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e52a9-ab9e-4902-9045-3bc526704bbe None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.686946 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.694990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 321/642] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:44.694990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e52a9-ab9e-4902-9045-3bc526704bbe None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-3e05f522-840a-49ec-b8db-85024503c99f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:44.697219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:44.698482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:44.700222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:44.700222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:44.700222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:44.704322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 322/643] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:44.705307 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05f522-840a-49ec-b8db-85024503c99f None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.786121 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.804464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e52a9-ab9e-4902-9045-3bc526704bbe None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.839504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71254451-d136-4db1-bdf9-ea32cc58c51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.839948 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e05f522-840a-49ec-b8db-85024503c99f None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.840105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71254451-d136-4db1-bdf9-ea32cc58c51a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.840382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71254451-d136-4db1-bdf9-ea32cc58c51a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.840655 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71254451-d136-4db1-bdf9-ea32cc58c51a None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.841287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71254451-d136-4db1-bdf9-ea32cc58c51a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:44.841586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71254451-d136-4db1-bdf9-ea32cc58c51a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.843019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71254451-d136-4db1-bdf9-ea32cc58c51a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.854468 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 322/644] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:44.855794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71254451-d136-4db1-bdf9-ea32cc58c51a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.882771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908e83a-7663-4d6c-a8d2-b6b78dc95d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.883728 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908e83a-7663-4d6c-a8d2-b6b78dc95d51 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.884017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908e83a-7663-4d6c-a8d2-b6b78dc95d51 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.884439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908e83a-7663-4d6c-a8d2-b6b78dc95d51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.891585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908e83a-7663-4d6c-a8d2-b6b78dc95d51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.893151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908e83a-7663-4d6c-a8d2-b6b78dc95d51 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-5908e83a-7663-4d6c-a8d2-b6b78dc95d51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:44.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:44.905688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:44.906854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:44.906854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:44.906854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:44.906854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:44.906854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:44.906854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:44.906854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:44.906854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:44.910511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 323/645] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:44.911320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908e83a-7663-4d6c-a8d2-b6b78dc95d51 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.924976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.968750 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71254451-d136-4db1-bdf9-ea32cc58c51a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.005849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8efa7-7186-4589-957c-f4b6ca1239e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.006457 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8efa7-7186-4589-957c-f4b6ca1239e2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.006741 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8efa7-7186-4589-957c-f4b6ca1239e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.007014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8efa7-7186-4589-957c-f4b6ca1239e2 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.007647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8efa7-7186-4589-957c-f4b6ca1239e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:45.007945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8efa7-7186-4589-957c-f4b6ca1239e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:45.009318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8efa7-7186-4589-957c-f4b6ca1239e2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:45.019615 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908e83a-7663-4d6c-a8d2-b6b78dc95d51 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.025329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 323/646] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:45.025329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8efa7-7186-4589-957c-f4b6ca1239e2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:45.037527 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:45.057501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e61d0-f7ec-4c4b-bf3d-bf775adb36b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.059268 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e61d0-f7ec-4c4b-bf3d-bf775adb36b3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.059750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e61d0-f7ec-4c4b-bf3d-bf775adb36b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.060208 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e61d0-f7ec-4c4b-bf3d-bf775adb36b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.074523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e61d0-f7ec-4c4b-bf3d-bf775adb36b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-c47e61d0-f7ec-4c4b-bf3d-bf775adb36b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:45.080679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:45.081651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:45.082540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:45.082540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:45.082540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:45.082540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:45.082540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:45.082540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:45.082540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:45.082540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:45.082540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:45.082540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:45.082540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:45.082540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:45.083169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 324/647] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:45.084455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e61d0-f7ec-4c4b-bf3d-bf775adb36b3 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:45.138426 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8efa7-7186-4589-957c-f4b6ca1239e2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.164184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:45.181820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0aad7-a2f9-4cf2-b316-0f174d2eb364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.182463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0aad7-a2f9-4cf2-b316-0f174d2eb364 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.182736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0aad7-a2f9-4cf2-b316-0f174d2eb364 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.183034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0aad7-a2f9-4cf2-b316-0f174d2eb364 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.189183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e61d0-f7ec-4c4b-bf3d-bf775adb36b3 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.191069 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0aad7-a2f9-4cf2-b316-0f174d2eb364 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:45.192603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0aad7-a2f9-4cf2-b316-0f174d2eb364 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-68d0aad7-a2f9-4cf2-b316-0f174d2eb364 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:45.204503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:45.205590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:45.206728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:45.206728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:45.206728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:45.206728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:45.206728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:45.206728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:45.206728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:45.206728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:45.206728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:45.206728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:45.208576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 324/648] 10.222.0.55 () {56 vars in 1093 bytes} [Thu Sep 8 13:39:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:45.209516 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0aad7-a2f9-4cf2-b316-0f174d2eb364 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:45.232446 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fa665-2794-4121-9689-988bf98c4fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.233051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fa665-2794-4121-9689-988bf98c4fdf None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.233327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fa665-2794-4121-9689-988bf98c4fdf None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.233593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fa665-2794-4121-9689-988bf98c4fdf None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.234315 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fa665-2794-4121-9689-988bf98c4fdf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:45.234621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fa665-2794-4121-9689-988bf98c4fdf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:45.236027 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fa665-2794-4121-9689-988bf98c4fdf None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:45.248072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 325/649] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:45.253451 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fa665-2794-4121-9689-988bf98c4fdf None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:45.295800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:45.318563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0aad7-a2f9-4cf2-b316-0f174d2eb364 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.354082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651c504-faf3-4af5-b6c8-8553a802a847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.354736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651c504-faf3-4af5-b6c8-8553a802a847 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.355679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651c504-faf3-4af5-b6c8-8553a802a847 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.355958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8651c504-faf3-4af5-b6c8-8553a802a847 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.363468 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651c504-faf3-4af5-b6c8-8553a802a847 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:45.364900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8651c504-faf3-4af5-b6c8-8553a802a847 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:45.365189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fa665-2794-4121-9689-988bf98c4fdf None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-8651c504-faf3-4af5-b6c8-8553a802a847 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:45.376413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:45.377555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:45.378545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:45.378545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:45.378545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:45.378545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:45.378545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:45.378545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:45.378545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:45.378545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:45.378545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:39:45.378545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:45.378545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 325/650] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:45.379150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651c504-faf3-4af5-b6c8-8553a802a847 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:45.409036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c555bfd-7700-4222-874a-d6df3d767687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.409951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c555bfd-7700-4222-874a-d6df3d767687 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.410407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c555bfd-7700-4222-874a-d6df3d767687 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.411096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c555bfd-7700-4222-874a-d6df3d767687 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.412612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c555bfd-7700-4222-874a-d6df3d767687 None admin] RBAC: Adding query filter params (name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:45.413204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c555bfd-7700-4222-874a-d6df3d767687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:45.415026 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c555bfd-7700-4222-874a-d6df3d767687 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:45.422737 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:45.425936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 326/651] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:45.426900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c555bfd-7700-4222-874a-d6df3d767687 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:45.486548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8651c504-faf3-4af5-b6c8-8553a802a847 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.524507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f83acd-a327-4455-ac6e-bf730598be7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.525272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f83acd-a327-4455-ac6e-bf730598be7f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.525551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f83acd-a327-4455-ac6e-bf730598be7f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.525868 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f83acd-a327-4455-ac6e-bf730598be7f None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.526576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f83acd-a327-4455-ac6e-bf730598be7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:45.527440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f83acd-a327-4455-ac6e-bf730598be7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:45.529094 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f83acd-a327-4455-ac6e-bf730598be7f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:45.534235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:45.537285 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c555bfd-7700-4222-874a-d6df3d767687 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.549774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 326/652] 10.222.0.55 () {56 vars in 1133 bytes} [Thu Sep 8 13:39:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:45.550787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f83acd-a327-4455-ac6e-bf730598be7f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:45.576363 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd609c-29ff-45c7-a5a2-68ffcda0a6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.577013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd609c-29ff-45c7-a5a2-68ffcda0a6ae None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.577263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd609c-29ff-45c7-a5a2-68ffcda0a6ae None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.577534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd609c-29ff-45c7-a5a2-68ffcda0a6ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.578278 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd609c-29ff-45c7-a5a2-68ffcda0a6ae None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=4e27f5b81c704dbebbd6a83b2713e237) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:45.578581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd609c-29ff-45c7-a5a2-68ffcda0a6ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:45.580100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd609c-29ff-45c7-a5a2-68ffcda0a6ae None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:45.585972 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 327/653] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:45 2022] GET /identity/v3/role_assignments?user.id=4e27f5b81c704dbebbd6a83b2713e237&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:45.586680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd609c-29ff-45c7-a5a2-68ffcda0a6ae None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:45.673496 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:45.680554 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f83acd-a327-4455-ac6e-bf730598be7f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.698511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd609c-29ff-45c7-a5a2-68ffcda0a6ae None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.719898 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa328ed-55dc-40cd-81eb-ab6006b11a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.720633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa328ed-55dc-40cd-81eb-ab6006b11a47 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.720796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa328ed-55dc-40cd-81eb-ab6006b11a47 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.721060 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa328ed-55dc-40cd-81eb-ab6006b11a47 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.721847 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa328ed-55dc-40cd-81eb-ab6006b11a47 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:45.722153 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa328ed-55dc-40cd-81eb-ab6006b11a47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:45.723621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa328ed-55dc-40cd-81eb-ab6006b11a47 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:45.743247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 327/654] 10.222.0.55 () {56 vars in 1137 bytes} [Thu Sep 8 13:39:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:45.744201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa328ed-55dc-40cd-81eb-ab6006b11a47 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:45.748225 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a71976-57fd-4cf6-9bba-43003291380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.748913 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a71976-57fd-4cf6-9bba-43003291380f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.749199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a71976-57fd-4cf6-9bba-43003291380f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.749474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a71976-57fd-4cf6-9bba-43003291380f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.761692 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a71976-57fd-4cf6-9bba-43003291380f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:45.763169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a71976-57fd-4cf6-9bba-43003291380f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-e2a71976-57fd-4cf6-9bba-43003291380f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:45.781347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:45.782366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:45.783275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:45.783275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:45.783275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:45.783275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 328/655] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:45.784592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bdde2-93e8-42e0-bc48-8e851bf6000e None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.784880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bdde2-93e8-42e0-bc48-8e851bf6000e None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.785154 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7bdde2-93e8-42e0-bc48-8e851bf6000e None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.786731 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 329/656] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:45.787679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7bdde2-93e8-42e0-bc48-8e851bf6000e None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:45.810699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:45.864452 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa328ed-55dc-40cd-81eb-ab6006b11a47 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.908583 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e8272-85fa-4652-a3da-8d8c8c944076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.909007 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e8272-85fa-4652-a3da-8d8c8c944076 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.909282 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e8272-85fa-4652-a3da-8d8c8c944076 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.909554 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9e8272-85fa-4652-a3da-8d8c8c944076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.909651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7bdde2-93e8-42e0-bc48-8e851bf6000e None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.936384 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:45.946509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fae6d-511e-441b-b56d-6ecab8e9f931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.947546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fae6d-511e-441b-b56d-6ecab8e9f931 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.948028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fae6d-511e-441b-b56d-6ecab8e9f931 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.948491 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fae6d-511e-441b-b56d-6ecab8e9f931 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.986928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e8272-85fa-4652-a3da-8d8c8c944076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:45.989024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9e8272-85fa-4652-a3da-8d8c8c944076 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:46.024996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fae6d-511e-441b-b56d-6ecab8e9f931 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:46.027359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fae6d-511e-441b-b56d-6ecab8e9f931 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:46.045862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:46.091651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 328/657] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:46.091651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e8272-85fa-4652-a3da-8d8c8c944076 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.139944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 330/658] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:46.141492 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2e993-ec14-4f36-b0d5-6254d8692f25 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.141791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2e993-ec14-4f36-b0d5-6254d8692f25 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.142071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2e993-ec14-4f36-b0d5-6254d8692f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.152517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:46.162435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-8ec2e993-ec14-4f36-b0d5-6254d8692f25 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:46.163598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8ec2e993-ec14-4f36-b0d5-6254d8692f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:46.199460 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9e8272-85fa-4652-a3da-8d8c8c944076 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:46.236117 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 331/659] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:46.236787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2e993-ec14-4f36-b0d5-6254d8692f25 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.241961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c7498-dcc8-416b-ab30-f8e40cb8c0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:46.242858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c7498-dcc8-416b-ab30-f8e40cb8c0af None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.243341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c7498-dcc8-416b-ab30-f8e40cb8c0af None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.243840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c7498-dcc8-416b-ab30-f8e40cb8c0af None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.245116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c7498-dcc8-416b-ab30-f8e40cb8c0af None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:46.246275 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c7498-dcc8-416b-ab30-f8e40cb8c0af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:46.248716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c7498-dcc8-416b-ab30-f8e40cb8c0af None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:46.263281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 329/660] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:46.264369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c7498-dcc8-416b-ab30-f8e40cb8c0af None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.288172 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:46.344188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2e993-ec14-4f36-b0d5-6254d8692f25 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:46.377194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c7498-dcc8-416b-ab30-f8e40cb8c0af None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:46.379059 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd6d9a-e602-4f68-83ac-a70f8f855369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:46.379765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd6d9a-e602-4f68-83ac-a70f8f855369 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.380115 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd6d9a-e602-4f68-83ac-a70f8f855369 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.380495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd6d9a-e602-4f68-83ac-a70f8f855369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.392146 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd6d9a-e602-4f68-83ac-a70f8f855369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:46.393840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd6d9a-e602-4f68-83ac-a70f8f855369 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:46.397577 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-edfd6d9a-e602-4f68-83ac-a70f8f855369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:46.409399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:46.410437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:46.411355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:46.411355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:46.411355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:46.411355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 332/661] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:46.411779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd6d9a-e602-4f68-83ac-a70f8f855369 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.421172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17101036-1244-4f62-9e8e-4bef369ab27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:46.422147 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17101036-1244-4f62-9e8e-4bef369ab27b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.422564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17101036-1244-4f62-9e8e-4bef369ab27b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.422981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17101036-1244-4f62-9e8e-4bef369ab27b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.435086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17101036-1244-4f62-9e8e-4bef369ab27b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:46.436765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17101036-1244-4f62-9e8e-4bef369ab27b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-17101036-1244-4f62-9e8e-4bef369ab27b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:46.453268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:46.454447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:46.455339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:46.455339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:46.455339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:46.461455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 330/662] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:46.462597 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17101036-1244-4f62-9e8e-4bef369ab27b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.515401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:46.519324 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd6d9a-e602-4f68-83ac-a70f8f855369 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:46.558808 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976aafb-bbcf-46e4-b9d0-5fa80e62ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:46.559447 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976aafb-bbcf-46e4-b9d0-5fa80e62ec04 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.559725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976aafb-bbcf-46e4-b9d0-5fa80e62ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.559996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976aafb-bbcf-46e4-b9d0-5fa80e62ec04 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.568146 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976aafb-bbcf-46e4-b9d0-5fa80e62ec04 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:46.569882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b976aafb-bbcf-46e4-b9d0-5fa80e62ec04 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:46.574246 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17101036-1244-4f62-9e8e-4bef369ab27b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-b976aafb-bbcf-46e4-b9d0-5fa80e62ec04 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:46.581095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:46.582195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:46.583240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:46.583240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:46.583240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:46.583240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:46.583240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:46.583240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:46.583240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:46.583240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:46.583240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:39:46.583240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:46.583240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 333/663] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:46.583851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976aafb-bbcf-46e4-b9d0-5fa80e62ec04 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.615600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e795573-abe6-4eea-ae9e-320533290ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:46.616453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e795573-abe6-4eea-ae9e-320533290ad5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.616920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e795573-abe6-4eea-ae9e-320533290ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.617192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e795573-abe6-4eea-ae9e-320533290ad5 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.617871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e795573-abe6-4eea-ae9e-320533290ad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:46.618173 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e795573-abe6-4eea-ae9e-320533290ad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:46.619525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e795573-abe6-4eea-ae9e-320533290ad5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:46.636193 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 331/664] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:46.636193 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e795573-abe6-4eea-ae9e-320533290ad5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.660757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:46.698997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976aafb-bbcf-46e4-b9d0-5fa80e62ec04 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:46.734290 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e795573-abe6-4eea-ae9e-320533290ad5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:46.736448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6698e4-c983-4139-8883-53ba598591fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:46.737052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6698e4-c983-4139-8883-53ba598591fb None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.737331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6698e4-c983-4139-8883-53ba598591fb None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.737597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6698e4-c983-4139-8883-53ba598591fb None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.738546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6698e4-c983-4139-8883-53ba598591fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:46.738546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6698e4-c983-4139-8883-53ba598591fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:46.739891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6698e4-c983-4139-8883-53ba598591fb None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:46.757052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 334/665] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:46.757052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6698e4-c983-4139-8883-53ba598591fb None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.775381 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:46.779869 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66359869-17d9-413e-a366-3f9c70f41d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:46.780527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66359869-17d9-413e-a366-3f9c70f41d2b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.780789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66359869-17d9-413e-a366-3f9c70f41d2b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.781047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66359869-17d9-413e-a366-3f9c70f41d2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.793790 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66359869-17d9-413e-a366-3f9c70f41d2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-66359869-17d9-413e-a366-3f9c70f41d2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:46.799027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:46.799937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:46.800804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:46.800804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:46.800804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:46.800804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:46.800804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:46.800804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:46.800804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:46.800804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:46.800804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:46.800804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:46.800804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:46.800804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:46.800804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 332/666] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:46.802002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66359869-17d9-413e-a366-3f9c70f41d2b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.869831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6698e4-c983-4139-8883-53ba598591fb None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:46.890215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:46.905658 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66359869-17d9-413e-a366-3f9c70f41d2b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:46.915819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66564b-0db5-4495-a0e0-e03790ac24d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:46.916424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66564b-0db5-4495-a0e0-e03790ac24d7 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.916701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66564b-0db5-4495-a0e0-e03790ac24d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.916975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66564b-0db5-4495-a0e0-e03790ac24d7 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.918292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66564b-0db5-4495-a0e0-e03790ac24d7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:46.918595 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66564b-0db5-4495-a0e0-e03790ac24d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:46.920061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66564b-0db5-4495-a0e0-e03790ac24d7 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:46.938302 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 335/667] 10.222.0.55 () {56 vars in 1139 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:46.939465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66564b-0db5-4495-a0e0-e03790ac24d7 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.952703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382f39f-e878-4417-bb1a-520c466542f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:46.953336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382f39f-e878-4417-bb1a-520c466542f0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.953611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382f39f-e878-4417-bb1a-520c466542f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.953912 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382f39f-e878-4417-bb1a-520c466542f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.960672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382f39f-e878-4417-bb1a-520c466542f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:46.962186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382f39f-e878-4417-bb1a-520c466542f0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1382f39f-e878-4417-bb1a-520c466542f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:46.974152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:46.975028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:46.980888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:46.980888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:46.980888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:46.980888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:46.980888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:46.980888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:46.980888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:46.980888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:46.980888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 333/668] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:46.980888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382f39f-e878-4417-bb1a-520c466542f0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:47.030906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:47.053925 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66564b-0db5-4495-a0e0-e03790ac24d7 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:47.090676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382f39f-e878-4417-bb1a-520c466542f0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:47.092438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22984a-a7f9-493a-af3d-fb94d479f645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:47.093184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22984a-a7f9-493a-af3d-fb94d479f645 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.093530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22984a-a7f9-493a-af3d-fb94d479f645 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.093895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22984a-a7f9-493a-af3d-fb94d479f645 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.100695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22984a-a7f9-493a-af3d-fb94d479f645 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:47.102223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22984a-a7f9-493a-af3d-fb94d479f645 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-bf22984a-a7f9-493a-af3d-fb94d479f645 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:47.117622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:47.118826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:47.119767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:47.119767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:47.119767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:47.119767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:47.119767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:47.119767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:47.119767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:47.119767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:47.119767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 336/669] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:47.121164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22984a-a7f9-493a-af3d-fb94d479f645 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:47.130011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138c3cd-f9da-4143-826d-69db1a7b88dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:47.130627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138c3cd-f9da-4143-826d-69db1a7b88dd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.130907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138c3cd-f9da-4143-826d-69db1a7b88dd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.131178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8138c3cd-f9da-4143-826d-69db1a7b88dd None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.131842 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138c3cd-f9da-4143-826d-69db1a7b88dd None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:47.132147 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138c3cd-f9da-4143-826d-69db1a7b88dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:47.133515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8138c3cd-f9da-4143-826d-69db1a7b88dd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:47.146983 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 334/670] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:47.146983 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138c3cd-f9da-4143-826d-69db1a7b88dd None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:47.159807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:47.238417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22984a-a7f9-493a-af3d-fb94d479f645 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:47.256432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8138c3cd-f9da-4143-826d-69db1a7b88dd None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:47.273500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0444847-404e-49f9-ac5b-5f33d21124e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:47.274577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0444847-404e-49f9-ac5b-5f33d21124e4 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.274948 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0444847-404e-49f9-ac5b-5f33d21124e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.275322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0444847-404e-49f9-ac5b-5f33d21124e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.280518 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:47.292154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c6e3b-3239-4a47-b044-494661a118b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:47.292954 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c6e3b-3239-4a47-b044-494661a118b4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.293235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c6e3b-3239-4a47-b044-494661a118b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.293506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c6e3b-3239-4a47-b044-494661a118b4 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.294225 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c6e3b-3239-4a47-b044-494661a118b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:47.294646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c6e3b-3239-4a47-b044-494661a118b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:47.295942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c6e3b-3239-4a47-b044-494661a118b4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:47.308239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 335/671] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:47.313649 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c6e3b-3239-4a47-b044-494661a118b4 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:47.351282 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0444847-404e-49f9-ac5b-5f33d21124e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:47.352724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0444847-404e-49f9-ac5b-5f33d21124e4 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:47.386249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:47.419052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c6e3b-3239-4a47-b044-494661a118b4 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:47.454154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f7d65-ed09-467c-a5a4-a7918e885a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:47.455201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f7d65-ed09-467c-a5a4-a7918e885a13 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.455483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f7d65-ed09-467c-a5a4-a7918e885a13 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.455918 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f7d65-ed09-467c-a5a4-a7918e885a13 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.456618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f7d65-ed09-467c-a5a4-a7918e885a13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:47.456930 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f7d65-ed09-467c-a5a4-a7918e885a13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:47.458403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f7d65-ed09-467c-a5a4-a7918e885a13 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:47.472483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 337/672] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:47.472483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0444847-404e-49f9-ac5b-5f33d21124e4 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:47.478167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 336/673] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:47.479839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f7d65-ed09-467c-a5a4-a7918e885a13 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:47.510526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:47.581855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0444847-404e-49f9-ac5b-5f33d21124e4 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:47.585147 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f7d65-ed09-467c-a5a4-a7918e885a13 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:47.615587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:47.618196 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fd75a-14d3-486d-9348-69bfc339cba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:47.618815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fd75a-14d3-486d-9348-69bfc339cba6 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.619098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fd75a-14d3-486d-9348-69bfc339cba6 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.619371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fd75a-14d3-486d-9348-69bfc339cba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.623186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a0df7-7ad3-4940-84c1-07efc4820ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:47.623794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a0df7-7ad3-4940-84c1-07efc4820ded None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.624076 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a0df7-7ad3-4940-84c1-07efc4820ded None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.624354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a0df7-7ad3-4940-84c1-07efc4820ded None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.625077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a0df7-7ad3-4940-84c1-07efc4820ded None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=6ffa73f81539472391983c51447af1b6) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:47.625387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a0df7-7ad3-4940-84c1-07efc4820ded None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:47.626840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3a0df7-7ad3-4940-84c1-07efc4820ded None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:47.629821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19fd75a-14d3-486d-9348-69bfc339cba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:47.631235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19fd75a-14d3-486d-9348-69bfc339cba6 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:47.633408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 337/674] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:47 2022] GET /identity/v3/role_assignments?user.id=6ffa73f81539472391983c51447af1b6&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:47.639341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a0df7-7ad3-4940-84c1-07efc4820ded None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-a19fd75a-14d3-486d-9348-69bfc339cba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:47.647013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:47.648054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:47.648980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:47.648980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:47.648980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:47.653894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 338/675] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:47.654899 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fd75a-14d3-486d-9348-69bfc339cba6 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:47.764772 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:47.782076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fd75a-14d3-486d-9348-69bfc339cba6 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:47.798637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a0df7-7ad3-4940-84c1-07efc4820ded None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:47.829021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426ed3b-d3f9-4a59-ba1e-d424e19d09c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:47.829256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426ed3b-d3f9-4a59-ba1e-d424e19d09c6 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.829481 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426ed3b-d3f9-4a59-ba1e-d424e19d09c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.829776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426ed3b-d3f9-4a59-ba1e-d424e19d09c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.830515 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426ed3b-d3f9-4a59-ba1e-d424e19d09c6 None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=ff93243f0c4c48d1924ea04a87a0e7c1) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:47.830825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426ed3b-d3f9-4a59-ba1e-d424e19d09c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:47.832259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426ed3b-d3f9-4a59-ba1e-d424e19d09c6 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:47.838839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 339/676] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:47 2022] GET /identity/v3/role_assignments?user.id=ff93243f0c4c48d1924ea04a87a0e7c1&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:47.840266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426ed3b-d3f9-4a59-ba1e-d424e19d09c6 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:47.846155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50a9be-e512-4202-b7b7-3d94917e2a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:47.847092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50a9be-e512-4202-b7b7-3d94917e2a71 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.847372 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50a9be-e512-4202-b7b7-3d94917e2a71 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.847644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50a9be-e512-4202-b7b7-3d94917e2a71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.860940 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50a9be-e512-4202-b7b7-3d94917e2a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:47.862465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50a9be-e512-4202-b7b7-3d94917e2a71 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-eb50a9be-e512-4202-b7b7-3d94917e2a71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:47.880640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:47.881782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:47.882729 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:47.882729 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:47.882729 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:47.884048 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 338/677] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:47.884465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac570b5-ca96-4d7f-a626-7351584f7ad9 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.884747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac570b5-ca96-4d7f-a626-7351584f7ad9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.885020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac570b5-ca96-4d7f-a626-7351584f7ad9 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.887120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 339/678] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:47.891978 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac570b5-ca96-4d7f-a626-7351584f7ad9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:47.966507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:47.987415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426ed3b-d3f9-4a59-ba1e-d424e19d09c6 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.028222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f838b-c467-41ce-ae97-a932ce3be24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.028896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f838b-c467-41ce-ae97-a932ce3be24f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.029180 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f838b-c467-41ce-ae97-a932ce3be24f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.029452 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f838b-c467-41ce-ae97-a932ce3be24f None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.030126 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f838b-c467-41ce-ae97-a932ce3be24f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:48.030430 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f838b-c467-41ce-ae97-a932ce3be24f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.031802 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f838b-c467-41ce-ae97-a932ce3be24f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:48.043969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac570b5-ca96-4d7f-a626-7351584f7ad9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.046793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 340/679] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:48.053336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e47b4e-4e66-4bb8-a9cf-29da43530da9 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.053836 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e47b4e-4e66-4bb8-a9cf-29da43530da9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.054277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e47b4e-4e66-4bb8-a9cf-29da43530da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.081665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-20e47b4e-4e66-4bb8-a9cf-29da43530da9 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:48.084185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-20e47b4e-4e66-4bb8-a9cf-29da43530da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:48.087758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf452e-256e-46d1-99c3-76fed6bca827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.088366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf452e-256e-46d1-99c3-76fed6bca827 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.088644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf452e-256e-46d1-99c3-76fed6bca827 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.088917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf452e-256e-46d1-99c3-76fed6bca827 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.090182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf452e-256e-46d1-99c3-76fed6bca827 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:48.090490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf452e-256e-46d1-99c3-76fed6bca827 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.091571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:48.092682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf452e-256e-46d1-99c3-76fed6bca827 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:48.107092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 340/680] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:48.109114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf452e-256e-46d1-99c3-76fed6bca827 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:48.163944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 341/681] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:48.164734 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e47b4e-4e66-4bb8-a9cf-29da43530da9 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:48.253584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf452e-256e-46d1-99c3-76fed6bca827 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.265715 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:48.314894 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cb725-e367-4517-a574-fbd2be110429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.315516 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cb725-e367-4517-a574-fbd2be110429 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.315791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cb725-e367-4517-a574-fbd2be110429 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.316058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656cb725-e367-4517-a574-fbd2be110429 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.328667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656cb725-e367-4517-a574-fbd2be110429 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.330853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656cb725-e367-4517-a574-fbd2be110429 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:48.346068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e47b4e-4e66-4bb8-a9cf-29da43530da9 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-656cb725-e367-4517-a574-fbd2be110429 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:48.352096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:48.353083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:48.354099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:48.354099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:48.354099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:48.354099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:48.354099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:48.354099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:48.354099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:48.354099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:48.354099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:48.354099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:48.354099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 341/682] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:48.361874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cb725-e367-4517-a574-fbd2be110429 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:48.405541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8418772-9b1a-47d0-9337-fa3dc6e1d3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.407019 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8418772-9b1a-47d0-9337-fa3dc6e1d3e3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.407556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8418772-9b1a-47d0-9337-fa3dc6e1d3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.409010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8418772-9b1a-47d0-9337-fa3dc6e1d3e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.419999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8418772-9b1a-47d0-9337-fa3dc6e1d3e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.421661 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8418772-9b1a-47d0-9337-fa3dc6e1d3e3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-b8418772-9b1a-47d0-9337-fa3dc6e1d3e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:48.444466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:48.445583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:48.446761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:48.446761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:48.446761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:48.446761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:48.446761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:48.446761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:48.446761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:48.446761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:48.450266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 342/683] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:48.455777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8418772-9b1a-47d0-9337-fa3dc6e1d3e3 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:48.463640 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:48.545276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656cb725-e367-4517-a574-fbd2be110429 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.581901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b7ecf-1518-491d-ad3c-32e64b476d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.582756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b7ecf-1518-491d-ad3c-32e64b476d54 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.583126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b7ecf-1518-491d-ad3c-32e64b476d54 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.583493 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b7ecf-1518-491d-ad3c-32e64b476d54 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.584311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b7ecf-1518-491d-ad3c-32e64b476d54 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:48.584711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b7ecf-1518-491d-ad3c-32e64b476d54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.586364 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b7ecf-1518-491d-ad3c-32e64b476d54 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:48.604903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8418772-9b1a-47d0-9337-fa3dc6e1d3e3 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.610014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 342/684] 10.222.0.55 () {56 vars in 1135 bytes} [Thu Sep 8 13:39:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:48.610014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b7ecf-1518-491d-ad3c-32e64b476d54 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:48.633576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:48.652332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59c7e1-83e6-46a2-8214-584a2aa9becb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.653082 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59c7e1-83e6-46a2-8214-584a2aa9becb None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.653454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59c7e1-83e6-46a2-8214-584a2aa9becb None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.653842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59c7e1-83e6-46a2-8214-584a2aa9becb None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.655006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59c7e1-83e6-46a2-8214-584a2aa9becb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:48.655426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59c7e1-83e6-46a2-8214-584a2aa9becb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.657535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59c7e1-83e6-46a2-8214-584a2aa9becb None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:48.673885 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 343/685] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:48.674997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59c7e1-83e6-46a2-8214-584a2aa9becb None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:48.739375 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b7ecf-1518-491d-ad3c-32e64b476d54 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.755963 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:48.780786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7a07e-7825-4763-876f-392bff0dbd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.781696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7a07e-7825-4763-876f-392bff0dbd86 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.782142 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7a07e-7825-4763-876f-392bff0dbd86 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.782503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7a07e-7825-4763-876f-392bff0dbd86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.793454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed7a07e-7825-4763-876f-392bff0dbd86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.795301 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed7a07e-7825-4763-876f-392bff0dbd86 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:48.804124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59c7e1-83e6-46a2-8214-584a2aa9becb None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-2ed7a07e-7825-4763-876f-392bff0dbd86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:48.810791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:48.811938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:48.813075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:48.813075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:48.813075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:48.820067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 343/686] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:48.820067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7a07e-7825-4763-876f-392bff0dbd86 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:48.853931 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa903da-8c2f-4f5c-a7a5-8e2fe0e7b7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.854742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa903da-8c2f-4f5c-a7a5-8e2fe0e7b7d4 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.855243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa903da-8c2f-4f5c-a7a5-8e2fe0e7b7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.855481 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa903da-8c2f-4f5c-a7a5-8e2fe0e7b7d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.925647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:48.945575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa903da-8c2f-4f5c-a7a5-8e2fe0e7b7d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.948032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa903da-8c2f-4f5c-a7a5-8e2fe0e7b7d4 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:48.950888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7a07e-7825-4763-876f-392bff0dbd86 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.984540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd00707-abaf-4df8-8794-8f9287d54929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.985295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd00707-abaf-4df8-8794-8f9287d54929 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.985672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd00707-abaf-4df8-8794-8f9287d54929 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.986062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd00707-abaf-4df8-8794-8f9287d54929 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.986878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd00707-abaf-4df8-8794-8f9287d54929 None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=531ebc79bf424e9b8762b2c7ec7fe8d0) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:48.987303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd00707-abaf-4df8-8794-8f9287d54929 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.989367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd00707-abaf-4df8-8794-8f9287d54929 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:49.002115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 344/687] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:48 2022] GET /identity/v3/role_assignments?user.id=531ebc79bf424e9b8762b2c7ec7fe8d0&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:49.003181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd00707-abaf-4df8-8794-8f9287d54929 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:49.020167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:49.072757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 344/688] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:49.080251 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa903da-8c2f-4f5c-a7a5-8e2fe0e7b7d4 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:49.205987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa903da-8c2f-4f5c-a7a5-8e2fe0e7b7d4 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:49.248191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec345cf5-10a5-4249-8c07-837ed1ba1aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:49.248430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd00707-abaf-4df8-8794-8f9287d54929 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:49.248837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec345cf5-10a5-4249-8c07-837ed1ba1aa5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:49.249121 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec345cf5-10a5-4249-8c07-837ed1ba1aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:49.249394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec345cf5-10a5-4249-8c07-837ed1ba1aa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:49.260284 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec345cf5-10a5-4249-8c07-837ed1ba1aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:49.261811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec345cf5-10a5-4249-8c07-837ed1ba1aa5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-ec345cf5-10a5-4249-8c07-837ed1ba1aa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:49.287536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:49.288564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:49.289496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:49.289496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:49.289496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:49.292929 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 345/689] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:49.298014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec345cf5-10a5-4249-8c07-837ed1ba1aa5 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:49.304779 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653cd02-30fa-4b01-8063-8f6dd82c5fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:49.305399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653cd02-30fa-4b01-8063-8f6dd82c5fc1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:49.305678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653cd02-30fa-4b01-8063-8f6dd82c5fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:49.305979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5653cd02-30fa-4b01-8063-8f6dd82c5fc1 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:49.306636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653cd02-30fa-4b01-8063-8f6dd82c5fc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:49.306979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653cd02-30fa-4b01-8063-8f6dd82c5fc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:49.308931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5653cd02-30fa-4b01-8063-8f6dd82c5fc1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:49.317753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:49.322159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:49.326998 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 345/690] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:49.332396 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653cd02-30fa-4b01-8063-8f6dd82c5fc1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:49.437100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec345cf5-10a5-4249-8c07-837ed1ba1aa5 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:49.464405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5653cd02-30fa-4b01-8063-8f6dd82c5fc1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:49.483857 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b31aa-ef50-45e9-a4d8-aab8c8af04c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:49.484533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b31aa-ef50-45e9-a4d8-aab8c8af04c1 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:49.484820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b31aa-ef50-45e9-a4d8-aab8c8af04c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:49.485094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b31aa-ef50-45e9-a4d8-aab8c8af04c1 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:49.486067 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b31aa-ef50-45e9-a4d8-aab8c8af04c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:49.486375 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b31aa-ef50-45e9-a4d8-aab8c8af04c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:49.488025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b31aa-ef50-45e9-a4d8-aab8c8af04c1 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:49.505632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 346/691] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:49.506803 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441f0c1-07f1-42d8-a366-a754567bb0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:49.507684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441f0c1-07f1-42d8-a366-a754567bb0a3 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:49.508213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441f0c1-07f1-42d8-a366-a754567bb0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:49.508693 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441f0c1-07f1-42d8-a366-a754567bb0a3 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:49.509577 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441f0c1-07f1-42d8-a366-a754567bb0a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:49.511620 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b31aa-ef50-45e9-a4d8-aab8c8af04c1 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:49.512574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441f0c1-07f1-42d8-a366-a754567bb0a3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:49.537567 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 346/692] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:39:49 2022] POST /identity/v3/users => generated 254 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:39:49.606090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:49.635109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b31aa-ef50-45e9-a4d8-aab8c8af04c1 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:49.675032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882cf95-d72a-4fb5-8251-3b8d71cf9ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:49.675672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882cf95-d72a-4fb5-8251-3b8d71cf9ba3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:49.675948 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882cf95-d72a-4fb5-8251-3b8d71cf9ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:49.676209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882cf95-d72a-4fb5-8251-3b8d71cf9ba3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:49.684538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882cf95-d72a-4fb5-8251-3b8d71cf9ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:49.686012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882cf95-d72a-4fb5-8251-3b8d71cf9ba3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-3882cf95-d72a-4fb5-8251-3b8d71cf9ba3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:49.699352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:49.700327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:49.701235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:49.701235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:49.701235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:49.701235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:49.701235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:49.701235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:49.701235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:49.701235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:49.701235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 347/693] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:49.709042 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441f0c1-07f1-42d8-a366-a754567bb0a3 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:49.771845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:49.828620 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441f0c1-07f1-42d8-a366-a754567bb0a3 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:49.865874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d2bff-fad6-46c5-bebc-91400e73aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:49.866492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d2bff-fad6-46c5-bebc-91400e73aa55 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:49.866772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d2bff-fad6-46c5-bebc-91400e73aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:49.867037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d2bff-fad6-46c5-bebc-91400e73aa55 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:49.867729 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d2bff-fad6-46c5-bebc-91400e73aa55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:49.868031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d2bff-fad6-46c5-bebc-91400e73aa55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:49.869525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d2bff-fad6-46c5-bebc-91400e73aa55 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:49.883699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 347/694] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:49.889582 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882cf95-d72a-4fb5-8251-3b8d71cf9ba3 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:50.004003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882cf95-d72a-4fb5-8251-3b8d71cf9ba3 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:50.044377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aedbb8-7ca4-4cf0-986f-0031189fab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:50.045234 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aedbb8-7ca4-4cf0-986f-0031189fab6f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:50.045653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aedbb8-7ca4-4cf0-986f-0031189fab6f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:50.046191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aedbb8-7ca4-4cf0-986f-0031189fab6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:50.047062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aedbb8-7ca4-4cf0-986f-0031189fab6f None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=4e5d73e8daa84975bc21e41e3cbef9cb) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:50.047532 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aedbb8-7ca4-4cf0-986f-0031189fab6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:50.049307 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aedbb8-7ca4-4cf0-986f-0031189fab6f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:50.051662 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:50.058735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 348/695] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:49 2022] GET /identity/v3/role_assignments?user.id=4e5d73e8daa84975bc21e41e3cbef9cb&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:50.240876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60590a1-c973-4ddf-8b74-f2ab4ffc755e None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:50.241330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60590a1-c973-4ddf-8b74-f2ab4ffc755e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:50.241706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60590a1-c973-4ddf-8b74-f2ab4ffc755e None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:50.243512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 348/696] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:50.249495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddac2a-d08f-4b25-b1b7-b5196c0f98b7 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:50.250376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddac2a-d08f-4b25-b1b7-b5196c0f98b7 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:50.250800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ddac2a-d08f-4b25-b1b7-b5196c0f98b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:50.280114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-79ddac2a-d08f-4b25-b1b7-b5196c0f98b7 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:50.281652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-79ddac2a-d08f-4b25-b1b7-b5196c0f98b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:50.364195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 349/697] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:50.409355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed76f8-b9bf-4571-97c3-3833696086a3 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:50.409890 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed76f8-b9bf-4571-97c3-3833696086a3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:50.409966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed76f8-b9bf-4571-97c3-3833696086a3 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:50.411255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 349/698] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:50.418016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6ec51-cb7a-4130-8667-fb2ca02cff00 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:50.418345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6ec51-cb7a-4130-8667-fb2ca02cff00 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:50.418626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b6ec51-cb7a-4130-8667-fb2ca02cff00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:50.438768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-81b6ec51-cb7a-4130-8667-fb2ca02cff00 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:50.439981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-81b6ec51-cb7a-4130-8667-fb2ca02cff00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:50.521893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 350/699] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:50.674441 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed76f8-b9bf-4571-97c3-3833696086a3 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:50.782734 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed76f8-b9bf-4571-97c3-3833696086a3 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:50.799762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:50.815700 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6ec51-cb7a-4130-8667-fb2ca02cff00 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:50.819937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50509318-27c0-4a25-b283-3fe4654c0a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:50.820626 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50509318-27c0-4a25-b283-3fe4654c0a91 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:50.820907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50509318-27c0-4a25-b283-3fe4654c0a91 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:50.821181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50509318-27c0-4a25-b283-3fe4654c0a91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:50.840809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50509318-27c0-4a25-b283-3fe4654c0a91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-50509318-27c0-4a25-b283-3fe4654c0a91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:50.846038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:50.846942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:50.847777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:50.847777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:50.847777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:50.847777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:50.847777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:50.847777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:50.847777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:50.847777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:50.847777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:50.847777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:50.847777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:50.847777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:50.856709 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 350/700] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:50.862286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50509318-27c0-4a25-b283-3fe4654c0a91 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:50.882918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:50.975588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b6ec51-cb7a-4130-8667-fb2ca02cff00 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:51.032785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50509318-27c0-4a25-b283-3fe4654c0a91 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:51.046415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f269c-bc52-44aa-8680-969693f2f40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:51.047102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f269c-bc52-44aa-8680-969693f2f40a None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.047383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f269c-bc52-44aa-8680-969693f2f40a None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.047706 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f269c-bc52-44aa-8680-969693f2f40a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.048297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f269c-bc52-44aa-8680-969693f2f40a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.049649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f269c-bc52-44aa-8680-969693f2f40a None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-458f269c-bc52-44aa-8680-969693f2f40a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:51.067575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:51.068653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:51.069784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 351/701] 10.222.0.55 () {56 vars in 1129 bytes} [Thu Sep 8 13:39:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:51.078297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f269c-bc52-44aa-8680-969693f2f40a None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:51.102870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b77d5-d3a4-40ab-858c-66d54a79b9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:51.103868 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b77d5-d3a4-40ab-858c-66d54a79b9cb None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.104370 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b77d5-d3a4-40ab-858c-66d54a79b9cb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.105387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b77d5-d3a4-40ab-858c-66d54a79b9cb None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.108050 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b77d5-d3a4-40ab-858c-66d54a79b9cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:51.108572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b77d5-d3a4-40ab-858c-66d54a79b9cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.110192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b77d5-d3a4-40ab-858c-66d54a79b9cb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.132167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 351/702] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:51.137370 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b77d5-d3a4-40ab-858c-66d54a79b9cb None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:51.206084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:51.252726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f269c-bc52-44aa-8680-969693f2f40a None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:51.303321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:51.329824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699eeab-e054-449f-9e3b-f4d903582eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:51.329824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699eeab-e054-449f-9e3b-f4d903582eb5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.329824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699eeab-e054-449f-9e3b-f4d903582eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.329824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699eeab-e054-449f-9e3b-f4d903582eb5 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.330272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699eeab-e054-449f-9e3b-f4d903582eb5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:51.330272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699eeab-e054-449f-9e3b-f4d903582eb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.331553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d699eeab-e054-449f-9e3b-f4d903582eb5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.343750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 352/703] 10.222.0.55 () {56 vars in 1124 bytes} [Thu Sep 8 13:39:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:51.357827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699eeab-e054-449f-9e3b-f4d903582eb5 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:51.371688 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b77d5-d3a4-40ab-858c-66d54a79b9cb None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:51.507107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d606fc-2f4d-484e-9794-37012f6ada27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:51.507714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d606fc-2f4d-484e-9794-37012f6ada27 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.508114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d606fc-2f4d-484e-9794-37012f6ada27 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.508218 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d606fc-2f4d-484e-9794-37012f6ada27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.545770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d606fc-2f4d-484e-9794-37012f6ada27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.545770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d606fc-2f4d-484e-9794-37012f6ada27 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-23d606fc-2f4d-484e-9794-37012f6ada27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:51.584641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:51.585699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:51.586680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:51.586680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:51.586680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:51.589944 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 352/704] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 453 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:51.608293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d606fc-2f4d-484e-9794-37012f6ada27 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:51.622519 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699eeab-e054-449f-9e3b-f4d903582eb5 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:51.673758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5861a-9808-4346-ab04-202673462c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:51.673758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5861a-9808-4346-ab04-202673462c02 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.673758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5861a-9808-4346-ab04-202673462c02 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.673758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5861a-9808-4346-ab04-202673462c02 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.674551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5861a-9808-4346-ab04-202673462c02 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:51.674551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5861a-9808-4346-ab04-202673462c02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.681544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c5861a-9808-4346-ab04-202673462c02 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.705470 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 353/705] 10.222.0.55 () {56 vars in 1124 bytes} [Thu Sep 8 13:39:51 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:51.884002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:51.920416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d606fc-2f4d-484e-9794-37012f6ada27 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:51.969256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932f3bc-2a30-41a3-9ea2-eaa76ae1faaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:51.969914 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932f3bc-2a30-41a3-9ea2-eaa76ae1faaf None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.970194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932f3bc-2a30-41a3-9ea2-eaa76ae1faaf None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.970467 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932f3bc-2a30-41a3-9ea2-eaa76ae1faaf None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.971118 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932f3bc-2a30-41a3-9ea2-eaa76ae1faaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:51.971425 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932f3bc-2a30-41a3-9ea2-eaa76ae1faaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.972793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932f3bc-2a30-41a3-9ea2-eaa76ae1faaf None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.983859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a6408-338f-464d-a43e-aa5d48e4a96c None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.984445 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a6408-338f-464d-a43e-aa5d48e4a96c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.985052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a6408-338f-464d-a43e-aa5d48e4a96c None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.985927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 353/706] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:51.987227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 354/707] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:51.991583 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932f3bc-2a30-41a3-9ea2-eaa76ae1faaf None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:51.993455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464d4cb-fd90-41a3-aa78-b4582df0bb03 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.994001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464d4cb-fd90-41a3-aa78-b4582df0bb03 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.994493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1464d4cb-fd90-41a3-aa78-b4582df0bb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:52.017850 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1464d4cb-fd90-41a3-aa78-b4582df0bb03 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:52.019391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1464d4cb-fd90-41a3-aa78-b4582df0bb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:52.088451 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:52.131246 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 355/708] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:39:52.151939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932f3bc-2a30-41a3-9ea2-eaa76ae1faaf None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:52.200609 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb5c40-cc12-4946-be86-45963297c565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:52.201253 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb5c40-cc12-4946-be86-45963297c565 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:52.201529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb5c40-cc12-4946-be86-45963297c565 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:52.201824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb5c40-cc12-4946-be86-45963297c565 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:52.214968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb5c40-cc12-4946-be86-45963297c565 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:52.216485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb5c40-cc12-4946-be86-45963297c565 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-91eb5c40-cc12-4946-be86-45963297c565 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:52.228662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:52.230227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:52.231186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:52.231186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:52.231186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:52.231186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:52.231186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:52.231186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:52.231186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:52.231186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:52.231186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:52.231186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:52.231186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 354/709] 10.222.0.55 () {56 vars in 1093 bytes} [Thu Sep 8 13:39:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:52.237603 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464d4cb-fd90-41a3-aa78-b4582df0bb03 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:52.372489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1464d4cb-fd90-41a3-aa78-b4582df0bb03 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:52.415890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1e17b-64d6-4581-b1a5-3f842b86ab9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:52.416416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1e17b-64d6-4581-b1a5-3f842b86ab9c None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:52.416688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1e17b-64d6-4581-b1a5-3f842b86ab9c None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:52.416971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee1e17b-64d6-4581-b1a5-3f842b86ab9c None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:52.417770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee1e17b-64d6-4581-b1a5-3f842b86ab9c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:52.418081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee1e17b-64d6-4581-b1a5-3f842b86ab9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:52.419537 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee1e17b-64d6-4581-b1a5-3f842b86ab9c None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:52.440863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 356/710] 10.222.0.55 () {56 vars in 1133 bytes} [Thu Sep 8 13:39:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:52.446826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb5c40-cc12-4946-be86-45963297c565 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:52.473303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:52.589770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1e17b-64d6-4581-b1a5-3f842b86ab9c None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:52.611884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb5c40-cc12-4946-be86-45963297c565 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:52.640942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:52.675838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69b25f-6217-4e46-b6af-07b7fc63c746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:52.676402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69b25f-6217-4e46-b6af-07b7fc63c746 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:52.676639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69b25f-6217-4e46-b6af-07b7fc63c746 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:52.676910 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69b25f-6217-4e46-b6af-07b7fc63c746 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:52.741270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee1e17b-64d6-4581-b1a5-3f842b86ab9c None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:52.779270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69b25f-6217-4e46-b6af-07b7fc63c746 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:52.780720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69b25f-6217-4e46-b6af-07b7fc63c746 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:52.792565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:52.794794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcedf3e-8fad-47fc-b1de-a083611d3e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:52.795895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcedf3e-8fad-47fc-b1de-a083611d3e8e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:52.797591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcedf3e-8fad-47fc-b1de-a083611d3e8e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:52.798494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcedf3e-8fad-47fc-b1de-a083611d3e8e None admin] PATH_INFO: `/v3/services/network` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:52.799237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcedf3e-8fad-47fc-b1de-a083611d3e8e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:52.801444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcedf3e-8fad-47fc-b1de-a083611d3e8e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-8fcedf3e-8fad-47fc-b1de-a083611d3e8e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:52.815102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:52.816377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:39:52.817483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:52.819236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 357/711] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:52 2022] GET /identity/v3/services/network => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:52.820421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefc4b9-9d76-4044-bbe3-95d11ceb5b1d None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:52.820703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefc4b9-9d76-4044-bbe3-95d11ceb5b1d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:52.820978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fefc4b9-9d76-4044-bbe3-95d11ceb5b1d None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:52.825784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 358/712] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:52.828842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefc4b9-9d76-4044-bbe3-95d11ceb5b1d None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:52.920101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 355/713] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:39:52.921368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10712b1-8cca-40c7-a152-2a240a444b12 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:52.921658 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10712b1-8cca-40c7-a152-2a240a444b12 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:52.921995 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10712b1-8cca-40c7-a152-2a240a444b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:52.945255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f10712b1-8cca-40c7-a152-2a240a444b12 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:52.947056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f10712b1-8cca-40c7-a152-2a240a444b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:52.955602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fefc4b9-9d76-4044-bbe3-95d11ceb5b1d None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:52.957604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:52.994164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c44e4-6c91-4fd8-842a-2323751ebc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:52.994771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c44e4-6c91-4fd8-842a-2323751ebc3a None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:52.995073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c44e4-6c91-4fd8-842a-2323751ebc3a None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:52.995883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c44e4-6c91-4fd8-842a-2323751ebc3a None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:52.996520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c44e4-6c91-4fd8-842a-2323751ebc3a None admin] RBAC: Adding query filter params (name=network) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:52.996824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c44e4-6c91-4fd8-842a-2323751ebc3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:52.998357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c44e4-6c91-4fd8-842a-2323751ebc3a None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:53.007630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 359/714] 10.222.0.55 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:53.008492 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c44e4-6c91-4fd8-842a-2323751ebc3a None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:53.034488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 356/715] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:53.035736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10712b1-8cca-40c7-a152-2a240a444b12 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:53.117597 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:53.134714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c44e4-6c91-4fd8-842a-2323751ebc3a None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:53.176196 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd1283-372c-4aed-8a04-05c55826fdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:53.177106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd1283-372c-4aed-8a04-05c55826fdb6 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.177583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd1283-372c-4aed-8a04-05c55826fdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.178026 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd1283-372c-4aed-8a04-05c55826fdb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.191287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd1283-372c-4aed-8a04-05c55826fdb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:53.193219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd1283-372c-4aed-8a04-05c55826fdb6 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:53.209305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10712b1-8cca-40c7-a152-2a240a444b12 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-77fd1283-372c-4aed-8a04-05c55826fdb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:53.211072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:53.212051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:53.212987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:53.212987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:53.212987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:53.215221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 360/716] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:53.220763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94a8c5-664e-4692-96c3-83cf1f377321 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.221049 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94a8c5-664e-4692-96c3-83cf1f377321 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.221325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94a8c5-664e-4692-96c3-83cf1f377321 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.226778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 361/717] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:53.227564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94a8c5-664e-4692-96c3-83cf1f377321 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:53.250965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:53.251597 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ee944-4789-4ec1-a768-becdd88de484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:53.252653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ee944-4789-4ec1-a768-becdd88de484 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.253144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ee944-4789-4ec1-a768-becdd88de484 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.253687 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ee944-4789-4ec1-a768-becdd88de484 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.254579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ee944-4789-4ec1-a768-becdd88de484 None admin] RBAC: Adding query filter params (type=network) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:53.255141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ee944-4789-4ec1-a768-becdd88de484 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:53.256703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ee944-4789-4ec1-a768-becdd88de484 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:53.263253 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 357/718] 10.222.0.55 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:53 2022] GET /identity/v3/services?type=network => generated 124 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:53.264769 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd488c-83d5-4ea5-bb12-1fc80b681905 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.265257 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd488c-83d5-4ea5-bb12-1fc80b681905 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.265770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd488c-83d5-4ea5-bb12-1fc80b681905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.302944 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-cfbd488c-83d5-4ea5-bb12-1fc80b681905 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:53.304650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cfbd488c-83d5-4ea5-bb12-1fc80b681905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:53.349405 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94a8c5-664e-4692-96c3-83cf1f377321 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:53.395285 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2845f-9f61-4b9b-8ff3-07c7deb53364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:53.395285 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2845f-9f61-4b9b-8ff3-07c7deb53364 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.395285 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2845f-9f61-4b9b-8ff3-07c7deb53364 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.395285 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2845f-9f61-4b9b-8ff3-07c7deb53364 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.396134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2845f-9f61-4b9b-8ff3-07c7deb53364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:53.396134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2845f-9f61-4b9b-8ff3-07c7deb53364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:53.396134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2845f-9f61-4b9b-8ff3-07c7deb53364 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:53.412593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 362/719] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:53.414980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2845f-9f61-4b9b-8ff3-07c7deb53364 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:53.437765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 358/720] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:53.473897 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd488c-83d5-4ea5-bb12-1fc80b681905 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:53.518115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:53.521415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:53.543013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2845f-9f61-4b9b-8ff3-07c7deb53364 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:53.582879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e16ca3-fb42-4451-b2ce-cbb60a73593c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:53.583818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e16ca3-fb42-4451-b2ce-cbb60a73593c None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.584215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd488c-83d5-4ea5-bb12-1fc80b681905 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:53.584508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e16ca3-fb42-4451-b2ce-cbb60a73593c None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.584955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e16ca3-fb42-4451-b2ce-cbb60a73593c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.593363 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e16ca3-fb42-4451-b2ce-cbb60a73593c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:53.595138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e16ca3-fb42-4451-b2ce-cbb60a73593c None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-76e16ca3-fb42-4451-b2ce-cbb60a73593c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:53.608435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:53.609342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:53.610273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:53.610273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:53.610273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:53.610273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:53.610273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:53.610273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:53.610273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:53.610273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:53.613272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 363/721] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:53.619823 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e16ca3-fb42-4451-b2ce-cbb60a73593c None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:53.623716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbd905-a83f-4095-bf4a-267b0d3f8b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:53.625337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbd905-a83f-4095-bf4a-267b0d3f8b1c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.625833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbd905-a83f-4095-bf4a-267b0d3f8b1c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.626185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbd905-a83f-4095-bf4a-267b0d3f8b1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.639543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbd905-a83f-4095-bf4a-267b0d3f8b1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-56fbd905-a83f-4095-bf4a-267b0d3f8b1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:53.646038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:53.646994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:53.647834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:53.647834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:53.647834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:53.647834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:53.647834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:53.647834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:53.647834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:53.647834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:53.647834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:53.647834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:53.647834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:53.647834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:53.649811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 359/722] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:53.653823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbd905-a83f-4095-bf4a-267b0d3f8b1c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:53.804285 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e16ca3-fb42-4451-b2ce-cbb60a73593c None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:53.824328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:53.832210 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbd905-a83f-4095-bf4a-267b0d3f8b1c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:53.846729 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95901885-38f1-4789-9ffa-b112598d9cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:53.847676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95901885-38f1-4789-9ffa-b112598d9cba None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.848212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95901885-38f1-4789-9ffa-b112598d9cba None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.848681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95901885-38f1-4789-9ffa-b112598d9cba None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.849586 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95901885-38f1-4789-9ffa-b112598d9cba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:53.850283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95901885-38f1-4789-9ffa-b112598d9cba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:53.852320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95901885-38f1-4789-9ffa-b112598d9cba None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:53.861597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:53.867357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 364/723] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:53.870682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95901885-38f1-4789-9ffa-b112598d9cba None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:53.874831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc3af8-713c-4257-bb3b-1feb5ba95736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:53.875785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc3af8-713c-4257-bb3b-1feb5ba95736 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.876359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc3af8-713c-4257-bb3b-1feb5ba95736 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.876839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc3af8-713c-4257-bb3b-1feb5ba95736 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.877713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc3af8-713c-4257-bb3b-1feb5ba95736 None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:53.878260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc3af8-713c-4257-bb3b-1feb5ba95736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:53.879835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc3af8-713c-4257-bb3b-1feb5ba95736 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:53.889424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 360/724] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:53.890507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc3af8-713c-4257-bb3b-1feb5ba95736 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:53.998941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95901885-38f1-4789-9ffa-b112598d9cba None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.002227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc3af8-713c-4257-bb3b-1feb5ba95736 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.038917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d071c7-c99a-48a7-98ef-c15d9b2aa38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:54.039710 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d071c7-c99a-48a7-98ef-c15d9b2aa38f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.040078 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d071c7-c99a-48a7-98ef-c15d9b2aa38f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.040424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d071c7-c99a-48a7-98ef-c15d9b2aa38f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.041042 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246d6c9-0225-4cd5-9774-3c900d0f329e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:54.041227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d071c7-c99a-48a7-98ef-c15d9b2aa38f None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=6ffa73f81539472391983c51447af1b6) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:54.041635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d071c7-c99a-48a7-98ef-c15d9b2aa38f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:54.042791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246d6c9-0225-4cd5-9774-3c900d0f329e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.043250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246d6c9-0225-4cd5-9774-3c900d0f329e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.043711 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246d6c9-0225-4cd5-9774-3c900d0f329e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.043900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d071c7-c99a-48a7-98ef-c15d9b2aa38f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:54.058050 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 361/725] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:53 2022] GET /identity/v3/role_assignments?user.id=6ffa73f81539472391983c51447af1b6&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:54.058050 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d071c7-c99a-48a7-98ef-c15d9b2aa38f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:54.076367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246d6c9-0225-4cd5-9774-3c900d0f329e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-8246d6c9-0225-4cd5-9774-3c900d0f329e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:54.086868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:54.087897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:54.088745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:54.088745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:54.088745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:54.088745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:54.088745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:54.088745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:54.088745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:54.088745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:54.088745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:54.088745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:54.088745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:54.088745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:54.090257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 365/726] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:54.100768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246d6c9-0225-4cd5-9774-3c900d0f329e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:54.117346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:54.221475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:54.264732 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d071c7-c99a-48a7-98ef-c15d9b2aa38f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.278066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246d6c9-0225-4cd5-9774-3c900d0f329e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.304848 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269e24a-ee09-414c-a1c6-e6ac63eaf294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:54.305899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269e24a-ee09-414c-a1c6-e6ac63eaf294 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.306408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269e24a-ee09-414c-a1c6-e6ac63eaf294 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.306879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269e24a-ee09-414c-a1c6-e6ac63eaf294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.319385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269e24a-ee09-414c-a1c6-e6ac63eaf294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:54.321255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cd2bf6-ddef-499f-ad92-3d513220d84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:54.321577 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269e24a-ee09-414c-a1c6-e6ac63eaf294 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:54.321977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd2bf6-ddef-499f-ad92-3d513220d84f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.322250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd2bf6-ddef-499f-ad92-3d513220d84f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.322520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd2bf6-ddef-499f-ad92-3d513220d84f None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.323591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd2bf6-ddef-499f-ad92-3d513220d84f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:54.323897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd2bf6-ddef-499f-ad92-3d513220d84f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:54.325424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cd2bf6-ddef-499f-ad92-3d513220d84f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:54.335861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 366/727] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:54.337229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda7397-8079-4439-854b-d8faa89fdd70 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.337514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda7397-8079-4439-854b-d8faa89fdd70 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.337820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda7397-8079-4439-854b-d8faa89fdd70 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-e269e24a-ee09-414c-a1c6-e6ac63eaf294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:54.340537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:54.341455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:54.342414 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:54.342414 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:54.342414 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:54.343689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 367/728] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:54.344477 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda7397-8079-4439-854b-d8faa89fdd70 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:54.345330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 362/729] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:54.346824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420367cc-282c-4581-8fc4-c35393f493b5 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.347328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420367cc-282c-4581-8fc4-c35393f493b5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.347809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420367cc-282c-4581-8fc4-c35393f493b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.366631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-420367cc-282c-4581-8fc4-c35393f493b5 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:54.368136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-420367cc-282c-4581-8fc4-c35393f493b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:54.441003 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 363/730] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:54.442662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420367cc-282c-4581-8fc4-c35393f493b5 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:54.447315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:54.475887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda7397-8079-4439-854b-d8faa89fdd70 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.512656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:54.541381 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bce3b6-bda6-4cbc-a97c-9de9876a609a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:54.542554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bce3b6-bda6-4cbc-a97c-9de9876a609a None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.542832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bce3b6-bda6-4cbc-a97c-9de9876a609a None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.543097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bce3b6-bda6-4cbc-a97c-9de9876a609a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.556074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420367cc-282c-4581-8fc4-c35393f493b5 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.568590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bce3b6-bda6-4cbc-a97c-9de9876a609a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:54.570675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bce3b6-bda6-4cbc-a97c-9de9876a609a None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-40bce3b6-bda6-4cbc-a97c-9de9876a609a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:54.598845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:54.599867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:54.600786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:54.600786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:54.600786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:54.600786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 368/731] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:54.601928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25eef6-3bf9-4db5-bcb6-2034e1de35bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:54.602781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25eef6-3bf9-4db5-bcb6-2034e1de35bf None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.603254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25eef6-3bf9-4db5-bcb6-2034e1de35bf None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.603729 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25eef6-3bf9-4db5-bcb6-2034e1de35bf None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.604604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25eef6-3bf9-4db5-bcb6-2034e1de35bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:54.605121 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25eef6-3bf9-4db5-bcb6-2034e1de35bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:54.606721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25eef6-3bf9-4db5-bcb6-2034e1de35bf None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:54.607810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bce3b6-bda6-4cbc-a97c-9de9876a609a None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:54.619277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 364/732] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:54.624455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25eef6-3bf9-4db5-bcb6-2034e1de35bf None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:54.739155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:54.798775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bce3b6-bda6-4cbc-a97c-9de9876a609a None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.805760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25eef6-3bf9-4db5-bcb6-2034e1de35bf None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.840262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c8124-488e-4bc9-8bd6-48c5d0452876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:54.840819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c8124-488e-4bc9-8bd6-48c5d0452876 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.841092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c8124-488e-4bc9-8bd6-48c5d0452876 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.841390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c8124-488e-4bc9-8bd6-48c5d0452876 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.844635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff2ef6-4f2e-472e-8321-15946db8a347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:54.845478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff2ef6-4f2e-472e-8321-15946db8a347 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.846016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff2ef6-4f2e-472e-8321-15946db8a347 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.846507 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff2ef6-4f2e-472e-8321-15946db8a347 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.847379 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff2ef6-4f2e-472e-8321-15946db8a347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:54.847902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff2ef6-4f2e-472e-8321-15946db8a347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:54.849298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c8124-488e-4bc9-8bd6-48c5d0452876 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:54.850043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff2ef6-4f2e-472e-8321-15946db8a347 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:54.850746 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65c8124-488e-4bc9-8bd6-48c5d0452876 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:54.863605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 369/733] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:54.864426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff2ef6-4f2e-472e-8321-15946db8a347 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-a65c8124-488e-4bc9-8bd6-48c5d0452876 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:54.869430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:54.871072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:54.872006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:54.872006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:54.872006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:54.872006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:54.872006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:54.872006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:54.872006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:54.872006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:54.872006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:54.872006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:54.874830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 365/734] 10.222.0.55 () {56 vars in 1117 bytes} [Thu Sep 8 13:39:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:54.876163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c8124-488e-4bc9-8bd6-48c5d0452876 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:54.907696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:54.987492 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff2ef6-4f2e-472e-8321-15946db8a347 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:54.988185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c8124-488e-4bc9-8bd6-48c5d0452876 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:55.024002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10169620-1568-4513-a073-eb74946622ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:55.025085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10169620-1568-4513-a073-eb74946622ec None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.025642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10169620-1568-4513-a073-eb74946622ec None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.026334 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10169620-1568-4513-a073-eb74946622ec None admin] PATH_INFO: `/v3/users/swift` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.029992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:55.032790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350290e1-2846-4bfc-80dd-0a304eda93bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:55.033417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350290e1-2846-4bfc-80dd-0a304eda93bf None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.033697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350290e1-2846-4bfc-80dd-0a304eda93bf None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.034028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350290e1-2846-4bfc-80dd-0a304eda93bf None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.034738 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350290e1-2846-4bfc-80dd-0a304eda93bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:55.036422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350290e1-2846-4bfc-80dd-0a304eda93bf None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:55.037287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10169620-1568-4513-a073-eb74946622ec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:55.039045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10169620-1568-4513-a073-eb74946622ec None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-10169620-1568-4513-a073-eb74946622ec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:55.053120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:55.054139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:55.055125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:55.055125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:55.055125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:55.055125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:55.055125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:55.055125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:55.055125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:55.055125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:55.055125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:55.055125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:55.057275 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 366/735] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:55.064523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 370/736] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:39:54 2022] POST /identity/v3/services => generated 251 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:55.064704 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10169620-1568-4513-a073-eb74946622ec None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:55.064785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350290e1-2846-4bfc-80dd-0a304eda93bf None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:55.186311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10169620-1568-4513-a073-eb74946622ec None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:55.240943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d676d-5a16-4192-8b63-2706f698bf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:55.241597 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d676d-5a16-4192-8b63-2706f698bf19 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.241913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d676d-5a16-4192-8b63-2706f698bf19 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.242400 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d676d-5a16-4192-8b63-2706f698bf19 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.243149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d676d-5a16-4192-8b63-2706f698bf19 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:55.243455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d676d-5a16-4192-8b63-2706f698bf19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:55.245157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d676d-5a16-4192-8b63-2706f698bf19 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:55.267799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 367/737] 10.222.0.55 () {56 vars in 1149 bytes} [Thu Sep 8 13:39:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:55.275649 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d676d-5a16-4192-8b63-2706f698bf19 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:55.296387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:55.307235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350290e1-2846-4bfc-80dd-0a304eda93bf None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:55.325323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:55.345963 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32dc4d-f447-44da-a981-9212451051a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:55.346578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32dc4d-f447-44da-a981-9212451051a5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.346860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32dc4d-f447-44da-a981-9212451051a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.347175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32dc4d-f447-44da-a981-9212451051a5 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.347916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32dc4d-f447-44da-a981-9212451051a5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:55.348266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32dc4d-f447-44da-a981-9212451051a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:55.349779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32dc4d-f447-44da-a981-9212451051a5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:55.371423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 371/738] 10.222.0.55 () {56 vars in 1135 bytes} [Thu Sep 8 13:39:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:55.376951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32dc4d-f447-44da-a981-9212451051a5 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:55.419736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d676d-5a16-4192-8b63-2706f698bf19 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:55.466858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c916c3e-09ca-4b8c-b0a1-50fcb4f5637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:55.468442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c916c3e-09ca-4b8c-b0a1-50fcb4f5637a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.468996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c916c3e-09ca-4b8c-b0a1-50fcb4f5637a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.469486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c916c3e-09ca-4b8c-b0a1-50fcb4f5637a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.500213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32dc4d-f447-44da-a981-9212451051a5 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:55.553519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c916c3e-09ca-4b8c-b0a1-50fcb4f5637a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:55.555344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c916c3e-09ca-4b8c-b0a1-50fcb4f5637a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:55.570547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2de8a1-2c41-4516-a2f5-b5a73b8d2c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:55.571177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2de8a1-2c41-4516-a2f5-b5a73b8d2c2d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.571458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2de8a1-2c41-4516-a2f5-b5a73b8d2c2d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.571730 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2de8a1-2c41-4516-a2f5-b5a73b8d2c2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.602352 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:55.614260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:55.701550 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 368/739] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:55 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:39:55.702345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2de8a1-2c41-4516-a2f5-b5a73b8d2c2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:55.703840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2de8a1-2c41-4516-a2f5-b5a73b8d2c2d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:55.718254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c916c3e-09ca-4b8c-b0a1-50fcb4f5637a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:55.834607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c916c3e-09ca-4b8c-b0a1-50fcb4f5637a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:55.883048 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 372/740] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:55 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:39:55.884505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672c3b3-dbe4-4e62-91da-37d6baca368d None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.884828 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0e68a-e9b8-4043-9696-2aae425df7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:55.884979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672c3b3-dbe4-4e62-91da-37d6baca368d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.885168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672c3b3-dbe4-4e62-91da-37d6baca368d None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.885479 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0e68a-e9b8-4043-9696-2aae425df7c3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.885836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0e68a-e9b8-4043-9696-2aae425df7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.886112 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0e68a-e9b8-4043-9696-2aae425df7c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.886670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 373/741] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:55.888090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672c3b3-dbe4-4e62-91da-37d6baca368d None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:55.889442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:55.910722 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0e68a-e9b8-4043-9696-2aae425df7c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:55.911872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0e68a-e9b8-4043-9696-2aae425df7c3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1dd0e68a-e9b8-4043-9696-2aae425df7c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:55.931543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:55.933101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:55.934045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:55.934045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:55.934045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:55.938106 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 369/742] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:55.939238 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd92de-927d-40f7-8fa1-ff6b132cbf23 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.939531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd92de-927d-40f7-8fa1-ff6b132cbf23 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.939810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd92de-927d-40f7-8fa1-ff6b132cbf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.957530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:55.959570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-edbd92de-927d-40f7-8fa1-ff6b132cbf23 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:55.960763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-edbd92de-927d-40f7-8fa1-ff6b132cbf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:56.016536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672c3b3-dbe4-4e62-91da-37d6baca368d None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.036748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 370/743] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:56.036748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd92de-927d-40f7-8fa1-ff6b132cbf23 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:56.070643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f47767-1fc7-4553-93db-35afabb4d017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:56.072771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f47767-1fc7-4553-93db-35afabb4d017 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.074059 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f47767-1fc7-4553-93db-35afabb4d017 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.074533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f47767-1fc7-4553-93db-35afabb4d017 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.096670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f47767-1fc7-4553-93db-35afabb4d017 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:56.099634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f47767-1fc7-4553-93db-35afabb4d017 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-60f47767-1fc7-4553-93db-35afabb4d017 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:56.127211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:56.128277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:56.129168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:56.129168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:56.129168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:56.132668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 374/744] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:56.138833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f47767-1fc7-4553-93db-35afabb4d017 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:56.255073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f47767-1fc7-4553-93db-35afabb4d017 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.256898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:56.263663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd92de-927d-40f7-8fa1-ff6b132cbf23 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.286797 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:56.298091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ddf40-6b3c-4d5e-a8f5-69ede4b9b085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:56.299122 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ddf40-6b3c-4d5e-a8f5-69ede4b9b085 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.299602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ddf40-6b3c-4d5e-a8f5-69ede4b9b085 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.300119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ddf40-6b3c-4d5e-a8f5-69ede4b9b085 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.300956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ddf40-6b3c-4d5e-a8f5-69ede4b9b085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:56.301454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ddf40-6b3c-4d5e-a8f5-69ede4b9b085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:56.303140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ddf40-6b3c-4d5e-a8f5-69ede4b9b085 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:56.319826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 375/745] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:56.330935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ddf40-6b3c-4d5e-a8f5-69ede4b9b085 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:56.345590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323224a-6614-4819-a15c-bcf186f97365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:56.346668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323224a-6614-4819-a15c-bcf186f97365 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.347195 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323224a-6614-4819-a15c-bcf186f97365 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.347721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4323224a-6614-4819-a15c-bcf186f97365 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.348710 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323224a-6614-4819-a15c-bcf186f97365 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:56.349224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323224a-6614-4819-a15c-bcf186f97365 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:56.351563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4323224a-6614-4819-a15c-bcf186f97365 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:56.367755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 371/746] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:56.369409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323224a-6614-4819-a15c-bcf186f97365 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:56.450610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ddf40-6b3c-4d5e-a8f5-69ede4b9b085 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.484490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4323224a-6614-4819-a15c-bcf186f97365 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.486603 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b52885-5c08-48c5-850f-1796a4cc975b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:56.487225 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b52885-5c08-48c5-850f-1796a4cc975b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.487500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b52885-5c08-48c5-850f-1796a4cc975b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.487766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b52885-5c08-48c5-850f-1796a4cc975b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.494538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b52885-5c08-48c5-850f-1796a4cc975b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:56.496024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b52885-5c08-48c5-850f-1796a4cc975b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-37b52885-5c08-48c5-850f-1796a4cc975b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:56.507091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:56.513312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:56.514501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:56.514501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:56.514501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:56.514501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:56.514501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:56.514501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:56.514501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:56.514501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:56.514501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 376/747] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:56.514501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b52885-5c08-48c5-850f-1796a4cc975b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:56.529942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe54ed-0ea0-46c0-a132-7b7da897844b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:56.530631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe54ed-0ea0-46c0-a132-7b7da897844b None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.530977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe54ed-0ea0-46c0-a132-7b7da897844b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.531330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe54ed-0ea0-46c0-a132-7b7da897844b None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.532048 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe54ed-0ea0-46c0-a132-7b7da897844b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:56.533477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe54ed-0ea0-46c0-a132-7b7da897844b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:56.538522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:56.552092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 372/748] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:39:56 2022] POST /identity/v3/services => generated 240 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:56.553397 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b1345-3eb6-40b2-ac80-1b17286bd33b None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.553685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b1345-3eb6-40b2-ac80-1b17286bd33b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.554027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b1345-3eb6-40b2-ac80-1b17286bd33b None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.555448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 373/749] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:56.556006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b1345-3eb6-40b2-ac80-1b17286bd33b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:56.635239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:56.646701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b52885-5c08-48c5-850f-1796a4cc975b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.700486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e139f-7dc0-43b9-93d3-993eaaabc957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:56.701146 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e139f-7dc0-43b9-93d3-993eaaabc957 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.701450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e139f-7dc0-43b9-93d3-993eaaabc957 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.702232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e139f-7dc0-43b9-93d3-993eaaabc957 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.710369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e139f-7dc0-43b9-93d3-993eaaabc957 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:56.711986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e139f-7dc0-43b9-93d3-993eaaabc957 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-899e139f-7dc0-43b9-93d3-993eaaabc957 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:56.726185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:56.727155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:56.728057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:56.728057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:56.728057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:56.728057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:56.728057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:56.728057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:56.728057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:56.728057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:56.728057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 377/750] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:56.729153 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b1345-3eb6-40b2-ac80-1b17286bd33b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.730191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5f8a1-6ae9-444e-a608-f3f5add63389 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.730562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5f8a1-6ae9-444e-a608-f3f5add63389 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.730906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5f8a1-6ae9-444e-a608-f3f5add63389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.751182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-77f5f8a1-6ae9-444e-a608-f3f5add63389 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:56.752500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-77f5f8a1-6ae9-444e-a608-f3f5add63389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:56.773082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f55ae1-c2f4-4bb6-8877-5c05ed013879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:56.774578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55ae1-c2f4-4bb6-8877-5c05ed013879 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.774578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55ae1-c2f4-4bb6-8877-5c05ed013879 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.774578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f55ae1-c2f4-4bb6-8877-5c05ed013879 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.775149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f55ae1-c2f4-4bb6-8877-5c05ed013879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:56.775529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f55ae1-c2f4-4bb6-8877-5c05ed013879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:56.777610 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f55ae1-c2f4-4bb6-8877-5c05ed013879 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:56.792008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 374/751] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:56.792732 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f55ae1-c2f4-4bb6-8877-5c05ed013879 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:56.829500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 378/752] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:56.830313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5f8a1-6ae9-444e-a608-f3f5add63389 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:56.841744 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:56.952690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:56.957683 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f55ae1-c2f4-4bb6-8877-5c05ed013879 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.988609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5f8a1-6ae9-444e-a608-f3f5add63389 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:57.018669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccea3a-6e97-4ded-b1ef-7fa5f9160261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:57.019278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccea3a-6e97-4ded-b1ef-7fa5f9160261 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.019551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccea3a-6e97-4ded-b1ef-7fa5f9160261 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.019817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ccea3a-6e97-4ded-b1ef-7fa5f9160261 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.020501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccea3a-6e97-4ded-b1ef-7fa5f9160261 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:57.020799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccea3a-6e97-4ded-b1ef-7fa5f9160261 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:57.023387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ccea3a-6e97-4ded-b1ef-7fa5f9160261 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:57.041315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 375/753] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:57.047519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccea3a-6e97-4ded-b1ef-7fa5f9160261 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:57.053028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b795a-a007-4e00-9f6a-6072a5fbfb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:57.053674 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b795a-a007-4e00-9f6a-6072a5fbfb7b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.053975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b795a-a007-4e00-9f6a-6072a5fbfb7b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.054237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b795a-a007-4e00-9f6a-6072a5fbfb7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.057119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b795a-a007-4e00-9f6a-6072a5fbfb7b None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=4e5d73e8daa84975bc21e41e3cbef9cb) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:57.057502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b795a-a007-4e00-9f6a-6072a5fbfb7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:57.058979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b795a-a007-4e00-9f6a-6072a5fbfb7b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:57.071938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 379/754] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:56 2022] GET /identity/v3/role_assignments?user.id=4e5d73e8daa84975bc21e41e3cbef9cb&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:57.162526 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ccea3a-6e97-4ded-b1ef-7fa5f9160261 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:57.215247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b795a-a007-4e00-9f6a-6072a5fbfb7b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:57.217768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65910a-949b-47c7-b6a4-538ad67a0761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:57.218661 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65910a-949b-47c7-b6a4-538ad67a0761 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.219149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65910a-949b-47c7-b6a4-538ad67a0761 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.219630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c65910a-949b-47c7-b6a4-538ad67a0761 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.220621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65910a-949b-47c7-b6a4-538ad67a0761 None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=41a5f117527a41fca00978c35fbd9eb1) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:57.221173 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65910a-949b-47c7-b6a4-538ad67a0761 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:57.222876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c65910a-949b-47c7-b6a4-538ad67a0761 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:57.230571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 376/755] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:57 2022] GET /identity/v3/role_assignments?user.id=41a5f117527a41fca00978c35fbd9eb1&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:57.343677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:57.383719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b795a-a007-4e00-9f6a-6072a5fbfb7b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:57.423356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f708c-646c-4b27-af7e-7ee266e4743b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:57.423990 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f708c-646c-4b27-af7e-7ee266e4743b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.424264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f708c-646c-4b27-af7e-7ee266e4743b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.424529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f708c-646c-4b27-af7e-7ee266e4743b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.437713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f708c-646c-4b27-af7e-7ee266e4743b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-f06f708c-646c-4b27-af7e-7ee266e4743b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:57.443229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:57.444486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:57.445336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:57.445336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:57.445336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:57.445336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:57.445336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:57.445336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:57.445336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:57.445336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:57.445336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:57.445336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:57.445336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:57.445336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:57.445336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 380/756] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:57.451333 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65910a-949b-47c7-b6a4-538ad67a0761 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:57.485755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:57.579622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c65910a-949b-47c7-b6a4-538ad67a0761 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:57.614386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4970e-26c3-41ce-9105-e733711db1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:57.615011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4970e-26c3-41ce-9105-e733711db1cf None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.615293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4970e-26c3-41ce-9105-e733711db1cf None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.615578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4970e-26c3-41ce-9105-e733711db1cf None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.616237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4970e-26c3-41ce-9105-e733711db1cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:57.616543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4970e-26c3-41ce-9105-e733711db1cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:57.617948 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4970e-26c3-41ce-9105-e733711db1cf None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:57.628037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 377/757] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:57.633773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f708c-646c-4b27-af7e-7ee266e4743b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:57.753612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f708c-646c-4b27-af7e-7ee266e4743b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:57.787252 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:57.789995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6cfe07-8fd7-4fbf-adc2-5d6940ff3e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:57.791005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cfe07-8fd7-4fbf-adc2-5d6940ff3e82 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.791284 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cfe07-8fd7-4fbf-adc2-5d6940ff3e82 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.791587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6cfe07-8fd7-4fbf-adc2-5d6940ff3e82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.803372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6cfe07-8fd7-4fbf-adc2-5d6940ff3e82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:57.804837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6cfe07-8fd7-4fbf-adc2-5d6940ff3e82 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-5b6cfe07-8fd7-4fbf-adc2-5d6940ff3e82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:57.819891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:57.820866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:57.821810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:57.821810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:57.821810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:57.821810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 381/758] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:57.827433 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4970e-26c3-41ce-9105-e733711db1cf None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:57.910834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5156ac7-d65e-400a-a72b-5a40e0ebc23a None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.910834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5156ac7-d65e-400a-a72b-5a40e0ebc23a None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.911090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5156ac7-d65e-400a-a72b-5a40e0ebc23a None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.913790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 382/759] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:57.917591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667739a-02b9-4364-8309-a9a11766a7c2 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.918157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667739a-02b9-4364-8309-a9a11766a7c2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.919169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667739a-02b9-4364-8309-a9a11766a7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.940517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-e667739a-02b9-4364-8309-a9a11766a7c2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:57.942116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e667739a-02b9-4364-8309-a9a11766a7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:57.946085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:57.983562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4970e-26c3-41ce-9105-e733711db1cf None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.052162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 383/760] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:58.054638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a1653-5047-4ef0-b0b4-a45a5f37a0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:58.055252 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a1653-5047-4ef0-b0b4-a45a5f37a0c5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:58.055532 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a1653-5047-4ef0-b0b4-a45a5f37a0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:58.055808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a1653-5047-4ef0-b0b4-a45a5f37a0c5 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:58.056458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a1653-5047-4ef0-b0b4-a45a5f37a0c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:58.056763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a1653-5047-4ef0-b0b4-a45a5f37a0c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:58.058682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a1653-5047-4ef0-b0b4-a45a5f37a0c5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.071323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 378/761] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:58.076862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667739a-02b9-4364-8309-a9a11766a7c2 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.179181 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667739a-02b9-4364-8309-a9a11766a7c2 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.194635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:58.220858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444e0f6-d1ae-439a-812e-304a707bc041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:58.221500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444e0f6-d1ae-439a-812e-304a707bc041 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:58.221806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444e0f6-d1ae-439a-812e-304a707bc041 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:58.222083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444e0f6-d1ae-439a-812e-304a707bc041 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:58.233517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444e0f6-d1ae-439a-812e-304a707bc041 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:58.234994 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444e0f6-d1ae-439a-812e-304a707bc041 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-0444e0f6-d1ae-439a-812e-304a707bc041 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:58.249280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:58.250298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:58.251267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:58.251267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:58.251267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:58.251267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:58.251267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:58.251267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:58.251267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:58.251267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:58.251267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:58.251267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:58.251267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 384/762] 10.222.0.55 () {56 vars in 1093 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:58.256089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a1653-5047-4ef0-b0b4-a45a5f37a0c5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.384485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a1653-5047-4ef0-b0b4-a45a5f37a0c5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.433357 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050923a-ec61-4be0-bed6-0fb83516fdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:58.433931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050923a-ec61-4be0-bed6-0fb83516fdc3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:58.434211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050923a-ec61-4be0-bed6-0fb83516fdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:58.434486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050923a-ec61-4be0-bed6-0fb83516fdc3 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:58.435219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050923a-ec61-4be0-bed6-0fb83516fdc3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:58.435527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050923a-ec61-4be0-bed6-0fb83516fdc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:58.436958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050923a-ec61-4be0-bed6-0fb83516fdc3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.465445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 379/763] 10.222.0.55 () {56 vars in 1133 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:58.470669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444e0f6-d1ae-439a-812e-304a707bc041 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.529014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050923a-ec61-4be0-bed6-0fb83516fdc3 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.532137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:58.597114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444e0f6-d1ae-439a-812e-304a707bc041 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.652867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58164aa6-3450-41e8-820b-d50eb9b7d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:58.653531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58164aa6-3450-41e8-820b-d50eb9b7d06b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:58.653778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58164aa6-3450-41e8-820b-d50eb9b7d06b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:58.655062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58164aa6-3450-41e8-820b-d50eb9b7d06b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:58.678381 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58164aa6-3450-41e8-820b-d50eb9b7d06b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:58.679837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58164aa6-3450-41e8-820b-d50eb9b7d06b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-58164aa6-3450-41e8-820b-d50eb9b7d06b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:58.709063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:58.710076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:58.710983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:58.710983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:58.710983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:58.710983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 385/764] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:58.719487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58164aa6-3450-41e8-820b-d50eb9b7d06b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.734416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050923a-ec61-4be0-bed6-0fb83516fdc3 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.771206 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826b55d-3acc-4692-a0cd-43ec9e67ab05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:58.771897 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826b55d-3acc-4692-a0cd-43ec9e67ab05 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:58.772184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826b55d-3acc-4692-a0cd-43ec9e67ab05 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:58.772458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826b55d-3acc-4692-a0cd-43ec9e67ab05 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:58.773045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826b55d-3acc-4692-a0cd-43ec9e67ab05 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:58.774453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826b55d-3acc-4692-a0cd-43ec9e67ab05 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-d826b55d-3acc-4692-a0cd-43ec9e67ab05 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:58.786939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:58.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:39:58.788796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 380/765] 10.222.0.55 () {56 vars in 1129 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:58.794436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826b55d-3acc-4692-a0cd-43ec9e67ab05 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.810348 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:58.879122 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:58.937829 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826b55d-3acc-4692-a0cd-43ec9e67ab05 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.954200 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58164aa6-3450-41e8-820b-d50eb9b7d06b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:59.006232 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66d3c8-46de-41ce-97af-bcd1f30bdbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:59.008102 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66d3c8-46de-41ce-97af-bcd1f30bdbc6 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.009209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66d3c8-46de-41ce-97af-bcd1f30bdbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.010068 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66d3c8-46de-41ce-97af-bcd1f30bdbc6 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.011883 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66d3c8-46de-41ce-97af-bcd1f30bdbc6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:59.012401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66d3c8-46de-41ce-97af-bcd1f30bdbc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:59.014379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66d3c8-46de-41ce-97af-bcd1f30bdbc6 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:59.018610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231247fa-d326-4dde-a0ea-77896b6ab7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:59.020082 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231247fa-d326-4dde-a0ea-77896b6ab7f6 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.020762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231247fa-d326-4dde-a0ea-77896b6ab7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.022222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231247fa-d326-4dde-a0ea-77896b6ab7f6 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.023070 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231247fa-d326-4dde-a0ea-77896b6ab7f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:59.023732 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231247fa-d326-4dde-a0ea-77896b6ab7f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:59.027553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231247fa-d326-4dde-a0ea-77896b6ab7f6 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:59.028046 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 381/766] 10.222.0.55 () {56 vars in 1124 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:59.035884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66d3c8-46de-41ce-97af-bcd1f30bdbc6 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:59.049062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 386/767] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:59.057133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231247fa-d326-4dde-a0ea-77896b6ab7f6 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:59.170465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:59.180416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66d3c8-46de-41ce-97af-bcd1f30bdbc6 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:59.189856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231247fa-d326-4dde-a0ea-77896b6ab7f6 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:59.212663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:59.220411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b50680-be0f-4a6a-a3f9-11fae4d17aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:59.221297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b50680-be0f-4a6a-a3f9-11fae4d17aac None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.221777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b50680-be0f-4a6a-a3f9-11fae4d17aac None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.222286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b50680-be0f-4a6a-a3f9-11fae4d17aac None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.223211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b50680-be0f-4a6a-a3f9-11fae4d17aac None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:59.223737 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b50680-be0f-4a6a-a3f9-11fae4d17aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:59.225337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b50680-be0f-4a6a-a3f9-11fae4d17aac None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:59.231077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 382/768] 10.222.0.55 () {56 vars in 1124 bytes} [Thu Sep 8 13:39:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:59.232676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f97f65-25fa-4153-9818-0803e4df413d None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.233164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f97f65-25fa-4153-9818-0803e4df413d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.233635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f97f65-25fa-4153-9818-0803e4df413d None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.239230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640fde04-738b-472d-a68d-ba03d35abd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:59.239230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fde04-738b-472d-a68d-ba03d35abd7c None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.239230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fde04-738b-472d-a68d-ba03d35abd7c None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.239230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640fde04-738b-472d-a68d-ba03d35abd7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.239794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 383/769] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:59.240920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f97f65-25fa-4153-9818-0803e4df413d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:59.246079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fde04-738b-472d-a68d-ba03d35abd7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:59.247530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640fde04-738b-472d-a68d-ba03d35abd7c None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-640fde04-738b-472d-a68d-ba03d35abd7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:59.262526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:59.263422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:59.264328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:59.264328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:59.264328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:59.264328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:59.264328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:59.264328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:59.264328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:59.264328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:39:59.264328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 387/770] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:59.265261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04126ed6-5392-4730-9365-74594c7508d6 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.265261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04126ed6-5392-4730-9365-74594c7508d6 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.265548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04126ed6-5392-4730-9365-74594c7508d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.295539 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-04126ed6-5392-4730-9365-74594c7508d6 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:59.296734 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-04126ed6-5392-4730-9365-74594c7508d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:59.372823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f97f65-25fa-4153-9818-0803e4df413d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:59.393786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 388/771] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:59.393786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04126ed6-5392-4730-9365-74594c7508d6 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:59.433766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179f009-fb8d-40d7-ae69-323c6d3be8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:59.434847 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179f009-fb8d-40d7-ae69-323c6d3be8cf None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.435380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179f009-fb8d-40d7-ae69-323c6d3be8cf None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.435872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179f009-fb8d-40d7-ae69-323c6d3be8cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.436920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179f009-fb8d-40d7-ae69-323c6d3be8cf None admin] RBAC: Adding query filter params (interface=public, service_id=f57ec9c3f898412a9f14f047b5704360, region_id=RegionOne) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:59.437454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179f009-fb8d-40d7-ae69-323c6d3be8cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:59.439171 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179f009-fb8d-40d7-ae69-323c6d3be8cf None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:59.458139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 384/772] 10.222.0.55 () {56 vars in 1251 bytes} [Thu Sep 8 13:39:59 2022] GET /identity/v3/endpoints?service_id=f57ec9c3f898412a9f14f047b5704360&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:59.464683 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179f009-fb8d-40d7-ae69-323c6d3be8cf None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:59.476838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:59.504893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04126ed6-5392-4730-9365-74594c7508d6 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:59.545627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5adbd-a4f5-43cf-ae4d-66e1c10ac00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:59.550585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5adbd-a4f5-43cf-ae4d-66e1c10ac00a None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.551077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5adbd-a4f5-43cf-ae4d-66e1c10ac00a None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.551549 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5adbd-a4f5-43cf-ae4d-66e1c10ac00a None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.552472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5adbd-a4f5-43cf-ae4d-66e1c10ac00a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:59.553012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5adbd-a4f5-43cf-ae4d-66e1c10ac00a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:59.556667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5adbd-a4f5-43cf-ae4d-66e1c10ac00a None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:59.585458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:59.587526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 389/773] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:59.591849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5adbd-a4f5-43cf-ae4d-66e1c10ac00a None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:59.678792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179f009-fb8d-40d7-ae69-323c6d3be8cf None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:59.748629 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1ec43-4fbd-4d0b-8545-75b13bb23758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:59.749665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1ec43-4fbd-4d0b-8545-75b13bb23758 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.750247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1ec43-4fbd-4d0b-8545-75b13bb23758 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.750725 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1ec43-4fbd-4d0b-8545-75b13bb23758 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.751598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1ec43-4fbd-4d0b-8545-75b13bb23758 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:59.753237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1ec43-4fbd-4d0b-8545-75b13bb23758 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:59.768016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 385/774] 10.222.0.55 () {56 vars in 1087 bytes} [Thu Sep 8 13:39:59 2022] GET /identity/v3/services => generated 772 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:59.769434 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951cd28-d751-429a-89b7-3e8421d4d4e9 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.769717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951cd28-d751-429a-89b7-3e8421d4d4e9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.770026 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951cd28-d751-429a-89b7-3e8421d4d4e9 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.771682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 386/775] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:59.777659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a57aa-12c9-4768-a9f7-e0b9ef3bf31b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.777990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a57aa-12c9-4768-a9f7-e0b9ef3bf31b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.778253 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a57aa-12c9-4768-a9f7-e0b9ef3bf31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.812889 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-764a57aa-12c9-4768-a9f7-e0b9ef3bf31b None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:59.814099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-764a57aa-12c9-4768-a9f7-e0b9ef3bf31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:59.900211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 387/776] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:59.901169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a57aa-12c9-4768-a9f7-e0b9ef3bf31b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:59.914581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5adbd-a4f5-43cf-ae4d-66e1c10ac00a None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:59.953815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63cfd2-aeac-4aed-8c22-75991905bbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:59.954601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63cfd2-aeac-4aed-8c22-75991905bbd3 None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.954738 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63cfd2-aeac-4aed-8c22-75991905bbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.955010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63cfd2-aeac-4aed-8c22-75991905bbd3 None admin] PATH_INFO: `/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/6ffa73f81539472391983c51447af1b6/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.957193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63cfd2-aeac-4aed-8c22-75991905bbd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe9c4e7bc14e70b57bf834cfff365f, user_id=6ffa73f81539472391983c51447af1b6, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:59.958533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63cfd2-aeac-4aed-8c22-75991905bbd3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:59.963016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:59.970542 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:59.981961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 390/777] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:59 2022] PUT /identity/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/6ffa73f81539472391983c51447af1b6/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:39:59.981961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e11e89-57e8-4c38-b64e-472a6094b970 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.981961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e11e89-57e8-4c38-b64e-472a6094b970 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.981961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e11e89-57e8-4c38-b64e-472a6094b970 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.983484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 391/778] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:59.988745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093eb7c1-19c7-4a08-9970-6cacc36071e3 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.989028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093eb7c1-19c7-4a08-9970-6cacc36071e3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.989303 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093eb7c1-19c7-4a08-9970-6cacc36071e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.013792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-093eb7c1-19c7-4a08-9970-6cacc36071e3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:00.015243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-093eb7c1-19c7-4a08-9970-6cacc36071e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:00.033851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a57aa-12c9-4768-a9f7-e0b9ef3bf31b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:00.073532 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48c86-f8ce-4daf-8e79-64f35ffee530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:00.074200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48c86-f8ce-4daf-8e79-64f35ffee530 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.074482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48c86-f8ce-4daf-8e79-64f35ffee530 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.074760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b48c86-f8ce-4daf-8e79-64f35ffee530 None admin] PATH_INFO: `/v3/services/network` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.075384 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b48c86-f8ce-4daf-8e79-64f35ffee530 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:00.076791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b48c86-f8ce-4daf-8e79-64f35ffee530 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-70b48c86-f8ce-4daf-8e79-64f35ffee530 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:00.086477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:00.087593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:00.088810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 388/779] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:59 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:00.093919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48c86-f8ce-4daf-8e79-64f35ffee530 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:00.114906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 392/780] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:00.221909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b48c86-f8ce-4daf-8e79-64f35ffee530 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:00.228143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093eb7c1-19c7-4a08-9970-6cacc36071e3 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:00.249068 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:00.271514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d39890-926b-449d-a16f-88594eae76b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:00.272465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d39890-926b-449d-a16f-88594eae76b3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.272943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d39890-926b-449d-a16f-88594eae76b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.273394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d39890-926b-449d-a16f-88594eae76b3 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.274237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d39890-926b-449d-a16f-88594eae76b3 None admin] RBAC: Adding query filter params (name=network) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:00.274774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d39890-926b-449d-a16f-88594eae76b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:00.277058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d39890-926b-449d-a16f-88594eae76b3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:00.291519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 389/781] 10.222.0.55 () {56 vars in 1112 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:00.298753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d39890-926b-449d-a16f-88594eae76b3 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:00.382607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093eb7c1-19c7-4a08-9970-6cacc36071e3 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:00.420724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:00.431564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b422054-b48b-41a9-b107-0d24a9a744e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:00.432465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b422054-b48b-41a9-b107-0d24a9a744e5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.432887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b422054-b48b-41a9-b107-0d24a9a744e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.433297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b422054-b48b-41a9-b107-0d24a9a744e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.445553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b422054-b48b-41a9-b107-0d24a9a744e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-1b422054-b48b-41a9-b107-0d24a9a744e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:00.450847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:00.451788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:00.452638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:00.452638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:00.452638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:00.452638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:00.452638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:00.452638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:00.452638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:00.452638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:00.452638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:00.452638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:00.452638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:00.452638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:00.455554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 393/782] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:00.464474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b422054-b48b-41a9-b107-0d24a9a744e5 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:00.487837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d39890-926b-449d-a16f-88594eae76b3 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:00.532580 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7899364-100a-4af6-b241-afcd125c064d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:00.533438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7899364-100a-4af6-b241-afcd125c064d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.533862 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7899364-100a-4af6-b241-afcd125c064d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.534302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7899364-100a-4af6-b241-afcd125c064d None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.535030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7899364-100a-4af6-b241-afcd125c064d None admin] RBAC: Adding query filter params (type=network) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:00.535429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7899364-100a-4af6-b241-afcd125c064d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:00.537074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7899364-100a-4af6-b241-afcd125c064d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:00.543167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 390/783] 10.222.0.55 () {56 vars in 1112 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:00.544087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7899364-100a-4af6-b241-afcd125c064d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:00.588448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b422054-b48b-41a9-b107-0d24a9a744e5 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:00.597161 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:00.640732 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa77acb-6dde-4447-b702-c144975c78c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:00.641644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa77acb-6dde-4447-b702-c144975c78c9 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.642226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa77acb-6dde-4447-b702-c144975c78c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.642502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa77acb-6dde-4447-b702-c144975c78c9 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.643186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa77acb-6dde-4447-b702-c144975c78c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:00.643490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa77acb-6dde-4447-b702-c144975c78c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:00.645629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa77acb-6dde-4447-b702-c144975c78c9 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:00.657898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 394/784] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:00.658756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa77acb-6dde-4447-b702-c144975c78c9 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:00.672101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7899364-100a-4af6-b241-afcd125c064d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:00.718101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:00.726265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4884a72-cc3e-460d-8afe-876d41cf7065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:00.727910 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4884a72-cc3e-460d-8afe-876d41cf7065 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.728402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4884a72-cc3e-460d-8afe-876d41cf7065 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.729654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4884a72-cc3e-460d-8afe-876d41cf7065 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.744798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4884a72-cc3e-460d-8afe-876d41cf7065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-c4884a72-cc3e-460d-8afe-876d41cf7065 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:00.752615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:00.753560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:00.754463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:00.754463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:00.754463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:00.754463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:00.754463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:00.754463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:00.754463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:00.754463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:00.754463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:00.754463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:00.754463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:40:00.754463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:00.758053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 391/785] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:00.759079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4884a72-cc3e-460d-8afe-876d41cf7065 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:00.780480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa77acb-6dde-4447-b702-c144975c78c9 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:00.817562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29514f-630f-4437-a3f9-4d45a9ccf6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:00.818204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29514f-630f-4437-a3f9-4d45a9ccf6df None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.818483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29514f-630f-4437-a3f9-4d45a9ccf6df None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.819034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29514f-630f-4437-a3f9-4d45a9ccf6df None admin] PATH_INFO: `/v3/endpoints` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.819723 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29514f-630f-4437-a3f9-4d45a9ccf6df None admin] RBAC: Adding query filter params (interface=public, service_id=3e37dc1685664fec875a949df34f6cc6, region_id=RegionOne) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:00.820027 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29514f-630f-4437-a3f9-4d45a9ccf6df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:00.821487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd29514f-630f-4437-a3f9-4d45a9ccf6df None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:00.833822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 395/786] 10.222.0.55 () {56 vars in 1251 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/endpoints?service_id=3e37dc1685664fec875a949df34f6cc6&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:00.833822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29514f-630f-4437-a3f9-4d45a9ccf6df None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:00.913626 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4884a72-cc3e-460d-8afe-876d41cf7065 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:00.916855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:00.953673 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e23cb0-0c2e-48c1-9a39-1b513349825f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:00.954677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e23cb0-0c2e-48c1-9a39-1b513349825f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.955235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e23cb0-0c2e-48c1-9a39-1b513349825f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.955840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e23cb0-0c2e-48c1-9a39-1b513349825f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.961819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29514f-630f-4437-a3f9-4d45a9ccf6df None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:00.967401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e23cb0-0c2e-48c1-9a39-1b513349825f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:00.969253 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e23cb0-0c2e-48c1-9a39-1b513349825f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:00.976494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-e1e23cb0-0c2e-48c1-9a39-1b513349825f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:00.988738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:00.989792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:00.990747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:00.990747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:00.990747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:00.997852 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 392/787] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:00.999335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e23cb0-0c2e-48c1-9a39-1b513349825f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.001546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a39bdd-beb9-4e20-ab88-ef99d1682397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.002232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a39bdd-beb9-4e20-ab88-ef99d1682397 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.002516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a39bdd-beb9-4e20-ab88-ef99d1682397 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.002786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a39bdd-beb9-4e20-ab88-ef99d1682397 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.003610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a39bdd-beb9-4e20-ab88-ef99d1682397 None admin] RBAC: Adding query filter params (name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:01.003858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a39bdd-beb9-4e20-ab88-ef99d1682397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.005577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a39bdd-beb9-4e20-ab88-ef99d1682397 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.019353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 396/788] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:01.020311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a39bdd-beb9-4e20-ab88-ef99d1682397 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.109559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e23cb0-0c2e-48c1-9a39-1b513349825f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.128057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a39bdd-beb9-4e20-ab88-ef99d1682397 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.146821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51fa50-b452-4c75-851d-ccd31493b118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.147708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51fa50-b452-4c75-851d-ccd31493b118 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.148209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51fa50-b452-4c75-851d-ccd31493b118 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.148684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51fa50-b452-4c75-851d-ccd31493b118 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.149601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51fa50-b452-4c75-851d-ccd31493b118 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.151263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51fa50-b452-4c75-851d-ccd31493b118 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.162164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 393/789] 10.222.0.55 () {56 vars in 1087 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:01.163291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51fa50-b452-4c75-851d-ccd31493b118 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.169270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb14be4-5fea-4b43-8e0f-07837027dded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.169969 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb14be4-5fea-4b43-8e0f-07837027dded None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.170247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb14be4-5fea-4b43-8e0f-07837027dded None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.170516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb14be4-5fea-4b43-8e0f-07837027dded None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.171200 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb14be4-5fea-4b43-8e0f-07837027dded None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:01.171504 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb14be4-5fea-4b43-8e0f-07837027dded None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.172908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb14be4-5fea-4b43-8e0f-07837027dded None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.206134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 397/790] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:01.211511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb14be4-5fea-4b43-8e0f-07837027dded None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.216474 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:01.286095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:01.362942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb14be4-5fea-4b43-8e0f-07837027dded None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.401788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51fa50-b452-4c75-851d-ccd31493b118 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.411795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f95ca5-879c-44de-a9e1-0ac923d7534c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.412494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f95ca5-879c-44de-a9e1-0ac923d7534c None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.412729 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f95ca5-879c-44de-a9e1-0ac923d7534c None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.412977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f95ca5-879c-44de-a9e1-0ac923d7534c None admin] PATH_INFO: `/v3/users/swift` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.422976 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f95ca5-879c-44de-a9e1-0ac923d7534c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.425062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f95ca5-879c-44de-a9e1-0ac923d7534c None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-03f95ca5-879c-44de-a9e1-0ac923d7534c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:40:01.444889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:01.445981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:01.446933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:01.446933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:01.446933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:40:01.446933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:40:01.446933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:40:01.446933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:40:01.446933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:40:01.446933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:40:01.446933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:40:01.446933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:01.446933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 398/791] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:01.452972 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f95ca5-879c-44de-a9e1-0ac923d7534c None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.462055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cce440-3ac7-4152-a168-1ad30ef89f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.463113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cce440-3ac7-4152-a168-1ad30ef89f60 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.463658 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cce440-3ac7-4152-a168-1ad30ef89f60 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.464173 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cce440-3ac7-4152-a168-1ad30ef89f60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.476107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cce440-3ac7-4152-a168-1ad30ef89f60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.477992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cce440-3ac7-4152-a168-1ad30ef89f60 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-35cce440-3ac7-4152-a168-1ad30ef89f60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:01.494323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:01.495893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:01.496809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:01.496809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:01.496809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:01.499002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 394/792] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:01.507351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cce440-3ac7-4152-a168-1ad30ef89f60 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.579106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:01.588516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f95ca5-879c-44de-a9e1-0ac923d7534c None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.594095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:01.626734 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cce440-3ac7-4152-a168-1ad30ef89f60 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.641184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7501f4-78b2-4a4d-ade6-c75f73e90021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.642046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7501f4-78b2-4a4d-ade6-c75f73e90021 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.642578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7501f4-78b2-4a4d-ade6-c75f73e90021 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.643007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7501f4-78b2-4a4d-ade6-c75f73e90021 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.643912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7501f4-78b2-4a4d-ade6-c75f73e90021 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:01.644368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7501f4-78b2-4a4d-ade6-c75f73e90021 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.645973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7501f4-78b2-4a4d-ade6-c75f73e90021 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.667741 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 399/793] 10.222.0.55 () {56 vars in 1135 bytes} [Thu Sep 8 13:40:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:01.674171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7501f4-78b2-4a4d-ade6-c75f73e90021 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.697946 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803a37e-036f-48cd-b0db-22eb3b1ecd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.699847 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803a37e-036f-48cd-b0db-22eb3b1ecd42 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.700717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803a37e-036f-48cd-b0db-22eb3b1ecd42 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.702395 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803a37e-036f-48cd-b0db-22eb3b1ecd42 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.703197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803a37e-036f-48cd-b0db-22eb3b1ecd42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:01.705159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803a37e-036f-48cd-b0db-22eb3b1ecd42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.706711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5803a37e-036f-48cd-b0db-22eb3b1ecd42 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.731706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 395/794] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:01.737274 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803a37e-036f-48cd-b0db-22eb3b1ecd42 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.844777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803a37e-036f-48cd-b0db-22eb3b1ecd42 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.887204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7501f4-78b2-4a4d-ade6-c75f73e90021 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.894794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:01.907320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5086f15-9486-4d91-bc0c-5eb0e4620c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.908211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5086f15-9486-4d91-bc0c-5eb0e4620c63 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.908639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5086f15-9486-4d91-bc0c-5eb0e4620c63 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.909238 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5086f15-9486-4d91-bc0c-5eb0e4620c63 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.922875 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5086f15-9486-4d91-bc0c-5eb0e4620c63 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.925120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5086f15-9486-4d91-bc0c-5eb0e4620c63 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.950550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e4d3d-fe55-44c4-a5a6-51999a65f17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-c5086f15-9486-4d91-bc0c-5eb0e4620c63 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:40:01.951264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:01.952194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:01.953166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:01.953166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:01.953166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:40:01.953166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:40:01.953166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:40:01.953166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:40:01.953166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:40:01.953166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:40:01.953166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:40:01.953166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:01.954212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e4d3d-fe55-44c4-a5a6-51999a65f17f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.954731 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e4d3d-fe55-44c4-a5a6-51999a65f17f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.955139 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e4d3d-fe55-44c4-a5a6-51999a65f17f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.962123 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 396/795] 10.222.0.55 () {56 vars in 1117 bytes} [Thu Sep 8 13:40:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:01.963103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5086f15-9486-4d91-bc0c-5eb0e4620c63 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.978078 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e4d3d-fe55-44c4-a5a6-51999a65f17f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.980197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e4d3d-fe55-44c4-a5a6-51999a65f17f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-578e4d3d-fe55-44c4-a5a6-51999a65f17f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:02.011329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:02.012350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:02.013286 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:02.013286 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:02.013286 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:02.014247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 400/796] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:02.020401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e4d3d-fe55-44c4-a5a6-51999a65f17f None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:02.025452 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:02.179061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e4d3d-fe55-44c4-a5a6-51999a65f17f None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:02.203959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5086f15-9486-4d91-bc0c-5eb0e4620c63 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:02.222408 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70655dbf-99fb-4603-b34a-beccb87e0438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:02.223706 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70655dbf-99fb-4603-b34a-beccb87e0438 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.225346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70655dbf-99fb-4603-b34a-beccb87e0438 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.225791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70655dbf-99fb-4603-b34a-beccb87e0438 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.226604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70655dbf-99fb-4603-b34a-beccb87e0438 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:02.227055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70655dbf-99fb-4603-b34a-beccb87e0438 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:02.228665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70655dbf-99fb-4603-b34a-beccb87e0438 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:02.242651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 401/797] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:02.249555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e4293-5795-41e2-9582-8687e80e6c55 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.250355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e4293-5795-41e2-9582-8687e80e6c55 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.250785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e4293-5795-41e2-9582-8687e80e6c55 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.252981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 402/798] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:02.253994 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e4293-5795-41e2-9582-8687e80e6c55 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:02.269622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5fc92-7d2e-4091-9234-5816384082a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:02.270280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5fc92-7d2e-4091-9234-5816384082a9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.270557 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5fc92-7d2e-4091-9234-5816384082a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.271084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5fc92-7d2e-4091-9234-5816384082a9 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.271834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5fc92-7d2e-4091-9234-5816384082a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:02.272138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5fc92-7d2e-4091-9234-5816384082a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:02.273777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5fc92-7d2e-4091-9234-5816384082a9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:02.297642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 397/799] 10.222.0.55 () {56 vars in 1149 bytes} [Thu Sep 8 13:40:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:02.300136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc6061-9529-46da-81f3-ac3d17907bf0 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.300420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc6061-9529-46da-81f3-ac3d17907bf0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.300686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcc6061-9529-46da-81f3-ac3d17907bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.322017 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0bcc6061-9529-46da-81f3-ac3d17907bf0 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:02.323806 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0bcc6061-9529-46da-81f3-ac3d17907bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:02.337065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:02.389985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:02.398116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20e4293-5795-41e2-9582-8687e80e6c55 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:02.419279 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 398/800] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:02.419279 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc6061-9529-46da-81f3-ac3d17907bf0 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:02.463805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430244e-6753-4d95-b7bc-3208af6e3803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:02.464492 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430244e-6753-4d95-b7bc-3208af6e3803 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.464864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430244e-6753-4d95-b7bc-3208af6e3803 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.465225 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430244e-6753-4d95-b7bc-3208af6e3803 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.472342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430244e-6753-4d95-b7bc-3208af6e3803 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:02.473905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430244e-6753-4d95-b7bc-3208af6e3803 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-9430244e-6753-4d95-b7bc-3208af6e3803 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:40:02.485313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:02.486412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:02.487529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:40:02.487529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:40:02.487529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:02.487529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:02.487529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:02.487529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:02.487529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:02.487529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:02.488442 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 403/801] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:02.494351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b0706-c089-4cbf-984c-ba4619f58b42 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.495223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b0706-c089-4cbf-984c-ba4619f58b42 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.495497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b0706-c089-4cbf-984c-ba4619f58b42 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.497158 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 404/802] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:02.497914 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b0706-c089-4cbf-984c-ba4619f58b42 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:02.546604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcc6061-9529-46da-81f3-ac3d17907bf0 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:02.591326 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0771c-dd7b-4549-86ed-3ec53b890095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:02.592416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0771c-dd7b-4549-86ed-3ec53b890095 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.593246 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0771c-dd7b-4549-86ed-3ec53b890095 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.593622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0771c-dd7b-4549-86ed-3ec53b890095 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.607741 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0771c-dd7b-4549-86ed-3ec53b890095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:02.609295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b0706-c089-4cbf-984c-ba4619f58b42 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:02.609420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0771c-dd7b-4549-86ed-3ec53b890095 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-a8a0771c-dd7b-4549-86ed-3ec53b890095 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:02.630149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:02.631304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:02.632368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:02.632368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:02.632368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:02.634136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 399/803] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:02.637332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd96a6c-59b6-4faf-b3de-ab77592abb33 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.638141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:02.638339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd96a6c-59b6-4faf-b3de-ab77592abb33 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.641952 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd96a6c-59b6-4faf-b3de-ab77592abb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.652081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d0dfe-7cc7-4e42-bf78-0f4547e35b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:02.652863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d0dfe-7cc7-4e42-bf78-0f4547e35b87 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.653231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d0dfe-7cc7-4e42-bf78-0f4547e35b87 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.653598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d0dfe-7cc7-4e42-bf78-0f4547e35b87 None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.654432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d0dfe-7cc7-4e42-bf78-0f4547e35b87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:02.654917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d0dfe-7cc7-4e42-bf78-0f4547e35b87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:02.656642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d0dfe-7cc7-4e42-bf78-0f4547e35b87 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:02.662285 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-acd96a6c-59b6-4faf-b3de-ab77592abb33 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:02.663487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-acd96a6c-59b6-4faf-b3de-ab77592abb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:02.673820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 405/804] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:40:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:02.674681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d0dfe-7cc7-4e42-bf78-0f4547e35b87 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:02.698583 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:02.754327 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 400/805] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:02.755382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd96a6c-59b6-4faf-b3de-ab77592abb33 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:02.789755 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d0dfe-7cc7-4e42-bf78-0f4547e35b87 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:02.835411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040a2fc-89af-4940-899e-5fe15218eb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:02.836769 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040a2fc-89af-4940-899e-5fe15218eb4f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.837277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040a2fc-89af-4940-899e-5fe15218eb4f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.837771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040a2fc-89af-4940-899e-5fe15218eb4f None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.839039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040a2fc-89af-4940-899e-5fe15218eb4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:02.839826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040a2fc-89af-4940-899e-5fe15218eb4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:02.841651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040a2fc-89af-4940-899e-5fe15218eb4f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:02.857283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 406/806] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:02.858825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040a2fc-89af-4940-899e-5fe15218eb4f None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:02.861618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd96a6c-59b6-4faf-b3de-ab77592abb33 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:02.897703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f9e0e-1a8f-41fa-b13d-51ac37c2300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:02.900958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f9e0e-1a8f-41fa-b13d-51ac37c2300a None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.901313 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f9e0e-1a8f-41fa-b13d-51ac37c2300a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.901809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f9e0e-1a8f-41fa-b13d-51ac37c2300a None admin] PATH_INFO: `/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/4e5d73e8daa84975bc21e41e3cbef9cb/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.904713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f9e0e-1a8f-41fa-b13d-51ac37c2300a None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe9c4e7bc14e70b57bf834cfff365f, user_id=4e5d73e8daa84975bc21e41e3cbef9cb, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:02.908229 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f9e0e-1a8f-41fa-b13d-51ac37c2300a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:02.916306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:02.933763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 401/807] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:40:02 2022] PUT /identity/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/4e5d73e8daa84975bc21e41e3cbef9cb/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:40:02.933763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f9e0e-1a8f-41fa-b13d-51ac37c2300a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:02.960957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:03.040074 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040a2fc-89af-4940-899e-5fe15218eb4f None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.062374 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f9e0e-1a8f-41fa-b13d-51ac37c2300a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.092649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092304f-6aa6-4647-ad2f-8648c5ed0258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.093546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092304f-6aa6-4647-ad2f-8648c5ed0258 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.093948 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092304f-6aa6-4647-ad2f-8648c5ed0258 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.094317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092304f-6aa6-4647-ad2f-8648c5ed0258 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.095332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092304f-6aa6-4647-ad2f-8648c5ed0258 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:03.097675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092304f-6aa6-4647-ad2f-8648c5ed0258 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:03.100944 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca44d83-9ae1-434f-ae79-1a57a5cdbd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.101628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca44d83-9ae1-434f-ae79-1a57a5cdbd7f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.101936 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca44d83-9ae1-434f-ae79-1a57a5cdbd7f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.102210 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca44d83-9ae1-434f-ae79-1a57a5cdbd7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-2092304f-6aa6-4647-ad2f-8648c5ed0258 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:03.109404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:03.110560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:40:03.111401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:03.112136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca44d83-9ae1-434f-ae79-1a57a5cdbd7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:03.112216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 407/808] 10.222.0.55 () {56 vars in 1129 bytes} [Thu Sep 8 13:40:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:03.112285 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca44d83-9ae1-434f-ae79-1a57a5cdbd7f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:03.112352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092304f-6aa6-4647-ad2f-8648c5ed0258 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-7ca44d83-9ae1-434f-ae79-1a57a5cdbd7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:40:03.132054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:03.133972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:03.136095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:40:03.136095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:40:03.136095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:03.136095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:03.136095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:03.136095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:03.136095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:03.136095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:03.136095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 402/809] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:03.136095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca44d83-9ae1-434f-ae79-1a57a5cdbd7f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:03.228876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:03.250337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092304f-6aa6-4647-ad2f-8648c5ed0258 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.250651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca44d83-9ae1-434f-ae79-1a57a5cdbd7f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.267445 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:03.299567 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdece024-44b0-4ff3-9d1c-86289288c6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.300095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdece024-44b0-4ff3-9d1c-86289288c6bc None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.300371 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdece024-44b0-4ff3-9d1c-86289288c6bc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.300643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdece024-44b0-4ff3-9d1c-86289288c6bc None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.301273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdece024-44b0-4ff3-9d1c-86289288c6bc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:03.301572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdece024-44b0-4ff3-9d1c-86289288c6bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:03.303030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdece024-44b0-4ff3-9d1c-86289288c6bc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:03.307521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d36a00-1e8e-4e02-8de8-076778d1c4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.308158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 403/810] 10.222.0.55 () {56 vars in 1124 bytes} [Thu Sep 8 13:40:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:03.308605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d36a00-1e8e-4e02-8de8-076778d1c4ba None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.309052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdece024-44b0-4ff3-9d1c-86289288c6bc None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:03.309341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d36a00-1e8e-4e02-8de8-076778d1c4ba None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.309861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d36a00-1e8e-4e02-8de8-076778d1c4ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.321751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d36a00-1e8e-4e02-8de8-076778d1c4ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-f5d36a00-1e8e-4e02-8de8-076778d1c4ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:03.328025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:03.328993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:03.329856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:03.329856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:03.329856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:03.329856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:03.329856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:03.329856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:03.329856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:03.329856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:03.329856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:03.329856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:03.329856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:03.329856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:03.332525 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 408/811] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:03.333499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d36a00-1e8e-4e02-8de8-076778d1c4ba None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:03.421216 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdece024-44b0-4ff3-9d1c-86289288c6bc None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.455877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e250ba-fe4d-4794-901f-7daf20684d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.456564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e250ba-fe4d-4794-901f-7daf20684d55 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.456909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e250ba-fe4d-4794-901f-7daf20684d55 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.457319 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e250ba-fe4d-4794-901f-7daf20684d55 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.458120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e250ba-fe4d-4794-901f-7daf20684d55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:03.458561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e250ba-fe4d-4794-901f-7daf20684d55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:03.459110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d36a00-1e8e-4e02-8de8-076778d1c4ba None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.460118 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e250ba-fe4d-4794-901f-7daf20684d55 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:03.476874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 404/812] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:40:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:03.478006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e250ba-fe4d-4794-901f-7daf20684d55 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:03.487752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:03.507031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4cd8f-4ff4-4f8f-a452-9876df3d9143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.508195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4cd8f-4ff4-4f8f-a452-9876df3d9143 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.508676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4cd8f-4ff4-4f8f-a452-9876df3d9143 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.509120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4cd8f-4ff4-4f8f-a452-9876df3d9143 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.509949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4cd8f-4ff4-4f8f-a452-9876df3d9143 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:03.510913 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4cd8f-4ff4-4f8f-a452-9876df3d9143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:03.512405 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4cd8f-4ff4-4f8f-a452-9876df3d9143 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:03.518402 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 409/813] 10.222.0.55 () {56 vars in 1124 bytes} [Thu Sep 8 13:40:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:03.519318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4cd8f-4ff4-4f8f-a452-9876df3d9143 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:03.554182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:03.599388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e250ba-fe4d-4794-901f-7daf20684d55 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.638623 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4cd8f-4ff4-4f8f-a452-9876df3d9143 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.645828 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247554e-0f78-4b72-9d55-09f69fabe509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.646550 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247554e-0f78-4b72-9d55-09f69fabe509 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.647350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247554e-0f78-4b72-9d55-09f69fabe509 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.647726 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247554e-0f78-4b72-9d55-09f69fabe509 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.648493 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247554e-0f78-4b72-9d55-09f69fabe509 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:03.648892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247554e-0f78-4b72-9d55-09f69fabe509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:03.651651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247554e-0f78-4b72-9d55-09f69fabe509 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:03.661952 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 405/814] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:03.666592 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247554e-0f78-4b72-9d55-09f69fabe509 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:03.681584 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc762c6-fae3-4b86-9697-1567595601e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.682303 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc762c6-fae3-4b86-9697-1567595601e4 None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.682695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc762c6-fae3-4b86-9697-1567595601e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.682992 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc762c6-fae3-4b86-9697-1567595601e4 None admin] PATH_INFO: `/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/41a5f117527a41fca00978c35fbd9eb1/roles/7377909b229e4449a4cbe1a22f8bf1f8` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.701324 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc762c6-fae3-4b86-9697-1567595601e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe9c4e7bc14e70b57bf834cfff365f, user_id=41a5f117527a41fca00978c35fbd9eb1, role_id=7377909b229e4449a4cbe1a22f8bf1f8)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:03.703065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc762c6-fae3-4b86-9697-1567595601e4 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:03.718609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 410/815] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:40:03 2022] PUT /identity/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/41a5f117527a41fca00978c35fbd9eb1/roles/7377909b229e4449a4cbe1a22f8bf1f8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:40:03.719496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc762c6-fae3-4b86-9697-1567595601e4 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:03.762490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:03.819250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247554e-0f78-4b72-9d55-09f69fabe509 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.830605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:03.854715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc762c6-fae3-4b86-9697-1567595601e4 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.880981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3075aa-615c-46d0-b2a7-83f37db89b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.882370 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3075aa-615c-46d0-b2a7-83f37db89b68 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.882947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3075aa-615c-46d0-b2a7-83f37db89b68 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.883661 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3075aa-615c-46d0-b2a7-83f37db89b68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.885279 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3075aa-615c-46d0-b2a7-83f37db89b68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:03.888387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3075aa-615c-46d0-b2a7-83f37db89b68 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:03.896629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcca570-0548-40b1-b15a-060eea4a4259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.897401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcca570-0548-40b1-b15a-060eea4a4259 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.897816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcca570-0548-40b1-b15a-060eea4a4259 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.898174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcca570-0548-40b1-b15a-060eea4a4259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.913888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcca570-0548-40b1-b15a-060eea4a4259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:03.915603 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcca570-0548-40b1-b15a-060eea4a4259 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:03.918914 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 406/816] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:40:03 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:03.920230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779dc3e-2bfe-4cc9-85c2-46b9b2b32118 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.920519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779dc3e-2bfe-4cc9-85c2-46b9b2b32118 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.920796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a779dc3e-2bfe-4cc9-85c2-46b9b2b32118 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.922318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 407/817] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:03.929930 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c80e8-3fc9-4994-ab01-e9ed3776fcf9 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.930308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c80e8-3fc9-4994-ab01-e9ed3776fcf9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.930653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c80e8-3fc9-4994-ab01-e9ed3776fcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-9dcca570-0548-40b1-b15a-060eea4a4259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:03.932983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:03.933955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:03.935076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:03.935076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:03.935076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:03.935076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 411/818] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:03.942511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcca570-0548-40b1-b15a-060eea4a4259 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:03.957232 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e44c80e8-3fc9-4994-ab01-e9ed3776fcf9 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:03.958560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e44c80e8-3fc9-4994-ab01-e9ed3776fcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:04.053894 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 408/819] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:04.080538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcca570-0548-40b1-b15a-060eea4a4259 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:04.119457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba4757-49c2-45e7-aa7e-b7adc6ecba14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:04.120471 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba4757-49c2-45e7-aa7e-b7adc6ecba14 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.120901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba4757-49c2-45e7-aa7e-b7adc6ecba14 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.121326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ba4757-49c2-45e7-aa7e-b7adc6ecba14 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.122166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba4757-49c2-45e7-aa7e-b7adc6ecba14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:04.122638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba4757-49c2-45e7-aa7e-b7adc6ecba14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:04.124192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ba4757-49c2-45e7-aa7e-b7adc6ecba14 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:04.126100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:04.139370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 412/820] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:04.145534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c80e8-3fc9-4994-ab01-e9ed3776fcf9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:04.202282 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:04.253538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c80e8-3fc9-4994-ab01-e9ed3776fcf9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:04.290821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8f747-15b7-485c-8ba6-a9c49ab1f3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:04.291543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8f747-15b7-485c-8ba6-a9c49ab1f3d3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.291825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8f747-15b7-485c-8ba6-a9c49ab1f3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.292099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8f747-15b7-485c-8ba6-a9c49ab1f3d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.299501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8f747-15b7-485c-8ba6-a9c49ab1f3d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:04.300944 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8f747-15b7-485c-8ba6-a9c49ab1f3d3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-a9a8f747-15b7-485c-8ba6-a9c49ab1f3d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:40:04.312763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:04.313718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:04.314719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:04.314719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:04.314719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:40:04.314719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:40:04.314719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:40:04.314719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:40:04.314719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:40:04.314719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:40:04.314719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:40:04.314719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:04.314719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 409/821] 10.222.0.55 () {56 vars in 1093 bytes} [Thu Sep 8 13:40:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:04.320124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba4757-49c2-45e7-aa7e-b7adc6ecba14 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:04.399891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8f747-15b7-485c-8ba6-a9c49ab1f3d3 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:04.429388 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ba4757-49c2-45e7-aa7e-b7adc6ecba14 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:04.468460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf18712-d5c1-47d9-85fe-c39a159bdb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:04.469075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf18712-d5c1-47d9-85fe-c39a159bdb16 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.469356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf18712-d5c1-47d9-85fe-c39a159bdb16 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.470023 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf18712-d5c1-47d9-85fe-c39a159bdb16 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.470476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf18712-d5c1-47d9-85fe-c39a159bdb16 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:04.470786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf18712-d5c1-47d9-85fe-c39a159bdb16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:04.477858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf18712-d5c1-47d9-85fe-c39a159bdb16 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:04.491850 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 413/822] 10.222.0.55 () {56 vars in 1133 bytes} [Thu Sep 8 13:40:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:04.498914 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf18712-d5c1-47d9-85fe-c39a159bdb16 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:04.559414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:04.563570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8f747-15b7-485c-8ba6-a9c49ab1f3d3 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:04.582308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:04.611881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf18712-d5c1-47d9-85fe-c39a159bdb16 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:04.627274 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4097a-448a-4d4e-b69c-e0db7042c0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:04.628281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4097a-448a-4d4e-b69c-e0db7042c0af None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.628408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4097a-448a-4d4e-b69c-e0db7042c0af None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.628604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4097a-448a-4d4e-b69c-e0db7042c0af None admin] PATH_INFO: `/v3/services/network` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.629206 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4097a-448a-4d4e-b69c-e0db7042c0af None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:04.631062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4097a-448a-4d4e-b69c-e0db7042c0af None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:04.657617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91fe8d-6dd7-478c-8b29-bc15f1c86726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-fae4097a-448a-4d4e-b69c-e0db7042c0af None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:04.658617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:04.659703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:40:04.660551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:04.661318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 410/823] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:04 2022] GET /identity/v3/services/network => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:04.662483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91fe8d-6dd7-478c-8b29-bc15f1c86726 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.662972 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91fe8d-6dd7-478c-8b29-bc15f1c86726 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.663444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91fe8d-6dd7-478c-8b29-bc15f1c86726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.668666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4097a-448a-4d4e-b69c-e0db7042c0af None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:04.738788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91fe8d-6dd7-478c-8b29-bc15f1c86726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:04.740592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91fe8d-6dd7-478c-8b29-bc15f1c86726 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:04.795833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:04.866663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 414/824] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:04 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:04.872387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91fe8d-6dd7-478c-8b29-bc15f1c86726 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:04.919136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4097a-448a-4d4e-b69c-e0db7042c0af None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:04.986041 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5378ba6-ed9e-4b3e-b63b-34d474192934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:04.986668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5378ba6-ed9e-4b3e-b63b-34d474192934 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.986927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5378ba6-ed9e-4b3e-b63b-34d474192934 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.989452 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5378ba6-ed9e-4b3e-b63b-34d474192934 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.990134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5378ba6-ed9e-4b3e-b63b-34d474192934 None admin] RBAC: Adding query filter params (name=network) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:04.990489 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5378ba6-ed9e-4b3e-b63b-34d474192934 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:04.991820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5378ba6-ed9e-4b3e-b63b-34d474192934 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.005349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 411/825] 10.222.0.55 () {56 vars in 1112 bytes} [Thu Sep 8 13:40:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:05.009417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5378ba6-ed9e-4b3e-b63b-34d474192934 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:05.063082 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91fe8d-6dd7-478c-8b29-bc15f1c86726 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.080716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:05.126415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a543f-98e5-4b38-a741-c6b9772fcb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.127753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a543f-98e5-4b38-a741-c6b9772fcb89 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.128259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a543f-98e5-4b38-a741-c6b9772fcb89 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.128728 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a543f-98e5-4b38-a741-c6b9772fcb89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.144188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a543f-98e5-4b38-a741-c6b9772fcb89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.146001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a543f-98e5-4b38-a741-c6b9772fcb89 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-325a543f-98e5-4b38-a741-c6b9772fcb89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:05.162280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:05.163250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:05.164182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:05.164182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:05.164182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:05.166292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 415/826] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:05.172656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a543f-98e5-4b38-a741-c6b9772fcb89 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:05.186272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:05.195896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5378ba6-ed9e-4b3e-b63b-34d474192934 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.267801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f94130-2075-4603-ac8e-de4d973abf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.268422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f94130-2075-4603-ac8e-de4d973abf21 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.268700 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f94130-2075-4603-ac8e-de4d973abf21 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.268976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f94130-2075-4603-ac8e-de4d973abf21 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.269679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f94130-2075-4603-ac8e-de4d973abf21 None admin] RBAC: Adding query filter params (type=network) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:05.270074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f94130-2075-4603-ac8e-de4d973abf21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.271512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f94130-2075-4603-ac8e-de4d973abf21 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.281357 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 412/827] 10.222.0.55 () {56 vars in 1112 bytes} [Thu Sep 8 13:40:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:05.287486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f94130-2075-4603-ac8e-de4d973abf21 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:05.367258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a543f-98e5-4b38-a741-c6b9772fcb89 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.429311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfce6d-55bb-4e0e-aaf0-e3d5eb7288e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.429908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfce6d-55bb-4e0e-aaf0-e3d5eb7288e8 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.430186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfce6d-55bb-4e0e-aaf0-e3d5eb7288e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.430454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfce6d-55bb-4e0e-aaf0-e3d5eb7288e8 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.431101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfce6d-55bb-4e0e-aaf0-e3d5eb7288e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:05.431402 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfce6d-55bb-4e0e-aaf0-e3d5eb7288e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.432798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfce6d-55bb-4e0e-aaf0-e3d5eb7288e8 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.459515 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 416/828] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:05.464795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfce6d-55bb-4e0e-aaf0-e3d5eb7288e8 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:05.469465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f94130-2075-4603-ac8e-de4d973abf21 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.471898 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:05.532354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ed745-54a9-43ee-8b08-68795580f1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.532981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ed745-54a9-43ee-8b08-68795580f1da None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.533262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ed745-54a9-43ee-8b08-68795580f1da None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.533535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ed745-54a9-43ee-8b08-68795580f1da None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.534716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ed745-54a9-43ee-8b08-68795580f1da None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.536465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ed745-54a9-43ee-8b08-68795580f1da None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.573609 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 413/829] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:40:05 2022] POST /identity/v3/endpoints => generated 337 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:05.582074 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:05.642615 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfce6d-55bb-4e0e-aaf0-e3d5eb7288e8 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.694254 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf76040-2737-4bc7-9f7e-257852ac96bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.694826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf76040-2737-4bc7-9f7e-257852ac96bd None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.695103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf76040-2737-4bc7-9f7e-257852ac96bd None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.695376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf76040-2737-4bc7-9f7e-257852ac96bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.703433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf76040-2737-4bc7-9f7e-257852ac96bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.704969 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf76040-2737-4bc7-9f7e-257852ac96bd None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-ecf76040-2737-4bc7-9f7e-257852ac96bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:40:05.718514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:05.719572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:05.720482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:40:05.720482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:40:05.720482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:05.720482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:05.720482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:05.720482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:05.720482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:05.720482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:05.720928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b155ef-e589-4407-9ce8-59c48a19610d None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.720928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b155ef-e589-4407-9ce8-59c48a19610d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.720928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b155ef-e589-4407-9ce8-59c48a19610d None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.721113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 417/830] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:05.722058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 414/831] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:05.727340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d493cb-3658-42ed-bf25-deceab53d82c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.727638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d493cb-3658-42ed-bf25-deceab53d82c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.727897 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d493cb-3658-42ed-bf25-deceab53d82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.730799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf76040-2737-4bc7-9f7e-257852ac96bd None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:05.748527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d2d493cb-3658-42ed-bf25-deceab53d82c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:05.749839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d2d493cb-3658-42ed-bf25-deceab53d82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:05.871030 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf76040-2737-4bc7-9f7e-257852ac96bd None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.875488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 415/832] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:05.895938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:05.920459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b80dd8-55d5-4d14-aeec-68e9b763a110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.921323 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b80dd8-55d5-4d14-aeec-68e9b763a110 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.921838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b80dd8-55d5-4d14-aeec-68e9b763a110 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.922321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b80dd8-55d5-4d14-aeec-68e9b763a110 None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.923227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b80dd8-55d5-4d14-aeec-68e9b763a110 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:05.923773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b80dd8-55d5-4d14-aeec-68e9b763a110 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.925398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b80dd8-55d5-4d14-aeec-68e9b763a110 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.940050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 418/833] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:40:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:05.949340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d493cb-3658-42ed-bf25-deceab53d82c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:06.057323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:06.071442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d493cb-3658-42ed-bf25-deceab53d82c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:06.180863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b80dd8-55d5-4d14-aeec-68e9b763a110 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:06.234538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021f0fd-e5fb-43d8-8077-a1e41c27e6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:06.235073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021f0fd-e5fb-43d8-8077-a1e41c27e6fa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.235348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021f0fd-e5fb-43d8-8077-a1e41c27e6fa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.235615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021f0fd-e5fb-43d8-8077-a1e41c27e6fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.236343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021f0fd-e5fb-43d8-8077-a1e41c27e6fa None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=6ffa73f81539472391983c51447af1b6) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:06.236650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021f0fd-e5fb-43d8-8077-a1e41c27e6fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:06.238117 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021f0fd-e5fb-43d8-8077-a1e41c27e6fa None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:06.245320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 416/834] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:05 2022] GET /identity/v3/role_assignments?user.id=6ffa73f81539472391983c51447af1b6&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:06.246549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43494a-9c6c-4e99-8379-ff0d221fda39 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.246829 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43494a-9c6c-4e99-8379-ff0d221fda39 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.247113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43494a-9c6c-4e99-8379-ff0d221fda39 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.248534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 417/835] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:06.255582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65feb00c-23f1-4fdf-86f9-9fcd2caf4ec9 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.256002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65feb00c-23f1-4fdf-86f9-9fcd2caf4ec9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.256370 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65feb00c-23f1-4fdf-86f9-9fcd2caf4ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.277887 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-65feb00c-23f1-4fdf-86f9-9fcd2caf4ec9 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:06.279202 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-65feb00c-23f1-4fdf-86f9-9fcd2caf4ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:06.313159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b80dd8-55d5-4d14-aeec-68e9b763a110 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:06.355018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4dfc6-83df-4ce0-8de0-535137818f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:06.355764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4dfc6-83df-4ce0-8de0-535137818f78 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.356134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4dfc6-83df-4ce0-8de0-535137818f78 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.356474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4dfc6-83df-4ce0-8de0-535137818f78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.368650 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4dfc6-83df-4ce0-8de0-535137818f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:06.373763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 418/836] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-36b4dfc6-83df-4ce0-8de0-535137818f78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:06.377765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:06.378733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:06.379597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:06.379597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:06.379597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:06.379597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:06.379597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:06.379597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:06.379597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:06.379597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:06.379597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:06.379597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:06.379597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:06.379597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:06.386997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 419/837] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:06.388668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65feb00c-23f1-4fdf-86f9-9fcd2caf4ec9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:06.483569 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:06.504353 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65feb00c-23f1-4fdf-86f9-9fcd2caf4ec9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:06.544184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c45b9-e82d-492e-9bdd-b73e6de22cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:06.544820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c45b9-e82d-492e-9bdd-b73e6de22cad None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.545099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c45b9-e82d-492e-9bdd-b73e6de22cad None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.545372 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c45b9-e82d-492e-9bdd-b73e6de22cad None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.546082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c45b9-e82d-492e-9bdd-b73e6de22cad None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:06.546386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c45b9-e82d-492e-9bdd-b73e6de22cad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:06.547776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c45b9-e82d-492e-9bdd-b73e6de22cad None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:06.560290 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 419/838] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:06.563322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4dfc6-83df-4ce0-8de0-535137818f78 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:06.610113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:06.637946 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0692c5a-10c1-4404-96b7-eab70527bab5 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.638753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0692c5a-10c1-4404-96b7-eab70527bab5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.639039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0692c5a-10c1-4404-96b7-eab70527bab5 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.641023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 420/839] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:06.646172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f0d85-60d4-41f2-8927-e6fc8df6bfca None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.646450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f0d85-60d4-41f2-8927-e6fc8df6bfca None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.646721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f0d85-60d4-41f2-8927-e6fc8df6bfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.666915 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-154f0d85-60d4-41f2-8927-e6fc8df6bfca None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:06.668104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-154f0d85-60d4-41f2-8927-e6fc8df6bfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:06.677414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4dfc6-83df-4ce0-8de0-535137818f78 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:06.715324 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31483f-230f-4e63-93a0-095f2eee62ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:06.716023 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31483f-230f-4e63-93a0-095f2eee62ae None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.716230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31483f-230f-4e63-93a0-095f2eee62ae None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.716496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31483f-230f-4e63-93a0-095f2eee62ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.727121 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31483f-230f-4e63-93a0-095f2eee62ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:06.728602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31483f-230f-4e63-93a0-095f2eee62ae None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:06.741827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 421/840] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-6f31483f-230f-4e63-93a0-095f2eee62ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:06.743721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:06.744652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:06.750361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:06.750361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:06.750361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:06.750361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 420/841] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:06.750599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f0d85-60d4-41f2-8927-e6fc8df6bfca None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:06.752004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31483f-230f-4e63-93a0-095f2eee62ae None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:06.830833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:06.864966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f0d85-60d4-41f2-8927-e6fc8df6bfca None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:06.881827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:06.893495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31483f-230f-4e63-93a0-095f2eee62ae None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:06.906500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c8cde-6514-4170-b023-24515e7fd0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:06.907169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c8cde-6514-4170-b023-24515e7fd0a8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.907446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c8cde-6514-4170-b023-24515e7fd0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.907718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c8cde-6514-4170-b023-24515e7fd0a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.918841 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c8cde-6514-4170-b023-24515e7fd0a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-361c8cde-6514-4170-b023-24515e7fd0a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:06.923707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:06.924630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:06.925535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:06.925535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:06.925535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:06.925535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:06.925535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:06.925535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:06.925535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:06.925535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:06.925535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:06.925535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:06.925535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:40:06.925535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:06.929694 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 422/842] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:40:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:06.931240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c8cde-6514-4170-b023-24515e7fd0a8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:06.945068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e2e22-eafe-4571-85ea-9ed3714f5498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:06.946174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e2e22-eafe-4571-85ea-9ed3714f5498 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.946653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e2e22-eafe-4571-85ea-9ed3714f5498 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.947522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e2e22-eafe-4571-85ea-9ed3714f5498 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.948672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e2e22-eafe-4571-85ea-9ed3714f5498 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:06.949216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e2e22-eafe-4571-85ea-9ed3714f5498 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:06.951249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e2e22-eafe-4571-85ea-9ed3714f5498 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:06.972915 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 421/843] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:06.974364 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e2e22-eafe-4571-85ea-9ed3714f5498 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:07.038601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c8cde-6514-4170-b023-24515e7fd0a8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:07.074382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17367297-8f95-4dfa-81ba-e5346177bfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:07.075141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17367297-8f95-4dfa-81ba-e5346177bfa8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:07.075660 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17367297-8f95-4dfa-81ba-e5346177bfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:07.075660 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17367297-8f95-4dfa-81ba-e5346177bfa8 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:07.076286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17367297-8f95-4dfa-81ba-e5346177bfa8 None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:07.076616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17367297-8f95-4dfa-81ba-e5346177bfa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:07.078076 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17367297-8f95-4dfa-81ba-e5346177bfa8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:07.079731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:07.089076 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 423/844] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:40:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:07.089843 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17367297-8f95-4dfa-81ba-e5346177bfa8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:07.097187 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e2e22-eafe-4571-85ea-9ed3714f5498 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:07.134122 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f339d-4982-4986-8a97-360768f86322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:07.134745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f339d-4982-4986-8a97-360768f86322 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:07.135084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f339d-4982-4986-8a97-360768f86322 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:07.135357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f339d-4982-4986-8a97-360768f86322 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:07.142948 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f339d-4982-4986-8a97-360768f86322 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:07.144368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f339d-4982-4986-8a97-360768f86322 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-351f339d-4982-4986-8a97-360768f86322 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:40:07.156297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:07.157437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:07.158427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:07.158427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:07.158427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:40:07.158427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:40:07.158427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:40:07.158427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:40:07.158427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:40:07.158427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:40:07.158427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:40:07.158427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:07.158427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 422/845] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:07.159043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f339d-4982-4986-8a97-360768f86322 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:07.177209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:07.203159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17367297-8f95-4dfa-81ba-e5346177bfa8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:07.239097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119504d-7c80-4c09-9fc8-1986913b5858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:07.239937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119504d-7c80-4c09-9fc8-1986913b5858 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:07.240215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119504d-7c80-4c09-9fc8-1986913b5858 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:07.240482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119504d-7c80-4c09-9fc8-1986913b5858 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:07.241087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119504d-7c80-4c09-9fc8-1986913b5858 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:07.242529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119504d-7c80-4c09-9fc8-1986913b5858 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-2119504d-7c80-4c09-9fc8-1986913b5858 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:07.251811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:07.252838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:07.253692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 424/846] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:40:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:07.254541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119504d-7c80-4c09-9fc8-1986913b5858 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:07.280103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f339d-4982-4986-8a97-360768f86322 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:07.317639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9bb7c-9faa-40f0-b62a-2ad45bc51259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:07.318391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:07.318844 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9bb7c-9faa-40f0-b62a-2ad45bc51259 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:07.319229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9bb7c-9faa-40f0-b62a-2ad45bc51259 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:07.319634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9bb7c-9faa-40f0-b62a-2ad45bc51259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:07.332394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9bb7c-9faa-40f0-b62a-2ad45bc51259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:07.333843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9bb7c-9faa-40f0-b62a-2ad45bc51259 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-05c9bb7c-9faa-40f0-b62a-2ad45bc51259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:07.351010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:07.351937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:07.353318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:07.353318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:07.353318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:07.353318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 423/847] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:07.354673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9bb7c-9faa-40f0-b62a-2ad45bc51259 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:07.364602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119504d-7c80-4c09-9fc8-1986913b5858 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:07.399658 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e919431-5e4e-4808-a42a-62beea31970b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:07.400261 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e919431-5e4e-4808-a42a-62beea31970b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:07.400533 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e919431-5e4e-4808-a42a-62beea31970b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:07.400825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e919431-5e4e-4808-a42a-62beea31970b None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:07.401538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e919431-5e4e-4808-a42a-62beea31970b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:07.401858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e919431-5e4e-4808-a42a-62beea31970b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:07.403270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e919431-5e4e-4808-a42a-62beea31970b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:07.419798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:07.422523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 425/848] 10.222.0.55 () {56 vars in 1135 bytes} [Thu Sep 8 13:40:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:07.423586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e919431-5e4e-4808-a42a-62beea31970b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:07.465931 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9bb7c-9faa-40f0-b62a-2ad45bc51259 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:07.501034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec4939-7547-485e-ae35-40b887394937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:07.501631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec4939-7547-485e-ae35-40b887394937 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:07.501935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec4939-7547-485e-ae35-40b887394937 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:07.502205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec4939-7547-485e-ae35-40b887394937 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:07.502829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec4939-7547-485e-ae35-40b887394937 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:07.503127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec4939-7547-485e-ae35-40b887394937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:07.504475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec4939-7547-485e-ae35-40b887394937 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:07.513813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 424/849] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:40:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:07.513813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec4939-7547-485e-ae35-40b887394937 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:07.536478 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e919431-5e4e-4808-a42a-62beea31970b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:07.554084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:07.571642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878a509-6e43-47c7-8cad-36ea3dbd8f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:07.572260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878a509-6e43-47c7-8cad-36ea3dbd8f1c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:07.573062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878a509-6e43-47c7-8cad-36ea3dbd8f1c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:07.573337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878a509-6e43-47c7-8cad-36ea3dbd8f1c None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:07.574017 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878a509-6e43-47c7-8cad-36ea3dbd8f1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:07.574320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878a509-6e43-47c7-8cad-36ea3dbd8f1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:07.575713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878a509-6e43-47c7-8cad-36ea3dbd8f1c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:07.592679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 426/850] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:07.592679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878a509-6e43-47c7-8cad-36ea3dbd8f1c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:07.617517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec4939-7547-485e-ae35-40b887394937 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:07.644767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:07.653680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f3bca-a2f4-4d3d-9cf0-f102a68daf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:07.654362 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f3bca-a2f4-4d3d-9cf0-f102a68daf67 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:07.654672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f3bca-a2f4-4d3d-9cf0-f102a68daf67 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:07.654896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f3bca-a2f4-4d3d-9cf0-f102a68daf67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:07.697152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878a509-6e43-47c7-8cad-36ea3dbd8f1c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:07.726806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f3bca-a2f4-4d3d-9cf0-f102a68daf67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:07.728242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f3bca-a2f4-4d3d-9cf0-f102a68daf67 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:07.730406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b7f7c-1d3e-4fbd-b208-70759121a706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:07.731071 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b7f7c-1d3e-4fbd-b208-70759121a706 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:07.731349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b7f7c-1d3e-4fbd-b208-70759121a706 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:07.731622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b7f7c-1d3e-4fbd-b208-70759121a706 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:07.732249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b7f7c-1d3e-4fbd-b208-70759121a706 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:07.732553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b7f7c-1d3e-4fbd-b208-70759121a706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:07.733920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b7f7c-1d3e-4fbd-b208-70759121a706 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:07.745765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 427/851] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:40:07 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:07.745765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b7f7c-1d3e-4fbd-b208-70759121a706 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:07.800902 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:07.852628 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 425/852] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:07 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:07.858817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f3bca-a2f4-4d3d-9cf0-f102a68daf67 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:07.876726 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b7f7c-1d3e-4fbd-b208-70759121a706 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:07.897421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:07.928814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96383c68-e87e-4d66-922c-9486b8ac6d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:07.929508 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96383c68-e87e-4d66-922c-9486b8ac6d58 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:07.929874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96383c68-e87e-4d66-922c-9486b8ac6d58 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:07.930356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96383c68-e87e-4d66-922c-9486b8ac6d58 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:07.940156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96383c68-e87e-4d66-922c-9486b8ac6d58 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:07.941586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96383c68-e87e-4d66-922c-9486b8ac6d58 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-96383c68-e87e-4d66-922c-9486b8ac6d58 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:40:07.955360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:07.956338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:07.957293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:07.957293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:07.957293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:40:07.957293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:40:07.957293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:40:07.957293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:40:07.957293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:40:07.957293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:40:07.957293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:40:07.957293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:07.957293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 428/853] 10.222.0.55 () {56 vars in 1117 bytes} [Thu Sep 8 13:40:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:07.963922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96383c68-e87e-4d66-922c-9486b8ac6d58 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:07.997062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f3bca-a2f4-4d3d-9cf0-f102a68daf67 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:08.041186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64745ce9-5971-4f2a-aa42-ee644cd6c8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:08.042004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64745ce9-5971-4f2a-aa42-ee644cd6c8c1 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:08.042289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64745ce9-5971-4f2a-aa42-ee644cd6c8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:08.042874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64745ce9-5971-4f2a-aa42-ee644cd6c8c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:08.058552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64745ce9-5971-4f2a-aa42-ee644cd6c8c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:08.060007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64745ce9-5971-4f2a-aa42-ee644cd6c8c1 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-64745ce9-5971-4f2a-aa42-ee644cd6c8c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:08.077178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:08.078207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:08.079120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:08.079120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:08.079120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:08.079120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 426/854] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:08.084761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64745ce9-5971-4f2a-aa42-ee644cd6c8c1 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:08.088016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96383c68-e87e-4d66-922c-9486b8ac6d58 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:08.090406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:08.134560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a74e3-92a0-4db5-9d86-a3000c8fa7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:08.136359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a74e3-92a0-4db5-9d86-a3000c8fa7ba None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:08.137789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a74e3-92a0-4db5-9d86-a3000c8fa7ba None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:08.138277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a74e3-92a0-4db5-9d86-a3000c8fa7ba None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:08.139219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a74e3-92a0-4db5-9d86-a3000c8fa7ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:08.139764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a74e3-92a0-4db5-9d86-a3000c8fa7ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:08.141459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a74e3-92a0-4db5-9d86-a3000c8fa7ba None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:08.172715 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 429/855] 10.222.0.55 () {56 vars in 1149 bytes} [Thu Sep 8 13:40:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:08.173852 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a74e3-92a0-4db5-9d86-a3000c8fa7ba None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:08.203691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:08.233667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64745ce9-5971-4f2a-aa42-ee644cd6c8c1 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:08.280056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a74e3-92a0-4db5-9d86-a3000c8fa7ba None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:08.283921 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaeeaed-d086-4349-aa93-0758e2cb4755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:08.285047 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaeeaed-d086-4349-aa93-0758e2cb4755 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:08.285499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaeeaed-d086-4349-aa93-0758e2cb4755 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:08.285962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaeeaed-d086-4349-aa93-0758e2cb4755 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:08.286762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaeeaed-d086-4349-aa93-0758e2cb4755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:08.287231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaeeaed-d086-4349-aa93-0758e2cb4755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:08.289194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaeeaed-d086-4349-aa93-0758e2cb4755 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:08.306300 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 427/856] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:08.312042 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaeeaed-d086-4349-aa93-0758e2cb4755 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:08.315861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37161e4-e7f5-4ca8-bb34-560da8dde6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:08.317313 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37161e4-e7f5-4ca8-bb34-560da8dde6ec None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:08.317313 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37161e4-e7f5-4ca8-bb34-560da8dde6ec None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:08.317313 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37161e4-e7f5-4ca8-bb34-560da8dde6ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:08.349077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:08.403162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37161e4-e7f5-4ca8-bb34-560da8dde6ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:08.405358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37161e4-e7f5-4ca8-bb34-560da8dde6ec None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:08.434474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaeeaed-d086-4349-aa93-0758e2cb4755 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:08.484413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e130ec-c26a-488e-97c3-8772604049df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:08.485582 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e130ec-c26a-488e-97c3-8772604049df None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:08.486036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e130ec-c26a-488e-97c3-8772604049df None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:08.487050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e130ec-c26a-488e-97c3-8772604049df None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:08.496343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e130ec-c26a-488e-97c3-8772604049df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:08.498495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e130ec-c26a-488e-97c3-8772604049df None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:08.502418 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-f0e130ec-c26a-488e-97c3-8772604049df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:40:08.516840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:08.517980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:08.518900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:40:08.518900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:40:08.518900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:08.518900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:08.518900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:08.518900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:08.518900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:08.518900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:08.519340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 428/857] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:08.526271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e130ec-c26a-488e-97c3-8772604049df None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:08.532796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 430/858] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:08 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:08.538511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37161e4-e7f5-4ca8-bb34-560da8dde6ec None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:08.606500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:08.635857 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e130ec-c26a-488e-97c3-8772604049df None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:08.656696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37161e4-e7f5-4ca8-bb34-560da8dde6ec None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:08.672046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a95de-d9cb-4759-ae19-a38f25124f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:08.672662 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a95de-d9cb-4759-ae19-a38f25124f03 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:08.672947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a95de-d9cb-4759-ae19-a38f25124f03 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:08.673223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a95de-d9cb-4759-ae19-a38f25124f03 None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:08.673947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a95de-d9cb-4759-ae19-a38f25124f03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:08.674257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a95de-d9cb-4759-ae19-a38f25124f03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:08.675691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a95de-d9cb-4759-ae19-a38f25124f03 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:08.696848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 429/859] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:40:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:08.696848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc047e-1517-4fec-814e-4e8409f95b32 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:08.696848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc047e-1517-4fec-814e-4e8409f95b32 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:08.696848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc047e-1517-4fec-814e-4e8409f95b32 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:08.696848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 430/860] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:08.699825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc047e-1517-4fec-814e-4e8409f95b32 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:08.703511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fb443-9203-4ab2-8535-1064ac22ffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:08.704597 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fb443-9203-4ab2-8535-1064ac22ffaa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:08.705093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fb443-9203-4ab2-8535-1064ac22ffaa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:08.705580 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fb443-9203-4ab2-8535-1064ac22ffaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:08.720057 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fb443-9203-4ab2-8535-1064ac22ffaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:08.722446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fb443-9203-4ab2-8535-1064ac22ffaa None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-3c1fb443-9203-4ab2-8535-1064ac22ffaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:08.741896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:08.743312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:08.744518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:08.744518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:08.744518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:08.748081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 431/861] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:08.749225 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e04491-00ec-4413-b23f-03eec5041a81 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:08.749506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e04491-00ec-4413-b23f-03eec5041a81 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:08.749880 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e04491-00ec-4413-b23f-03eec5041a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:08.772060 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-65e04491-00ec-4413-b23f-03eec5041a81 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:08.773023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-65e04491-00ec-4413-b23f-03eec5041a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:08.785522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:08.835826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc047e-1517-4fec-814e-4e8409f95b32 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:08.842441 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 432/862] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:08.843060 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e04491-00ec-4413-b23f-03eec5041a81 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:08.878983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ed663-16c6-4468-bbeb-0561c87254fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:08.879869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ed663-16c6-4468-bbeb-0561c87254fd None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:08.880341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ed663-16c6-4468-bbeb-0561c87254fd None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:08.880757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ed663-16c6-4468-bbeb-0561c87254fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:08.881621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ed663-16c6-4468-bbeb-0561c87254fd None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=4e5d73e8daa84975bc21e41e3cbef9cb) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:08.882284 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ed663-16c6-4468-bbeb-0561c87254fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:08.884471 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ed663-16c6-4468-bbeb-0561c87254fd None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:08.902225 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 431/863] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:08 2022] GET /identity/v3/role_assignments?user.id=4e5d73e8daa84975bc21e41e3cbef9cb&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:08.910297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:08.981325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e04491-00ec-4413-b23f-03eec5041a81 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:09.019437 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc59dd9-2d80-4a02-8629-90ac1400e6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:09.020350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59dd9-2d80-4a02-8629-90ac1400e6ed None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:09.020779 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59dd9-2d80-4a02-8629-90ac1400e6ed None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:09.021214 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59dd9-2d80-4a02-8629-90ac1400e6ed None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:09.022056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc59dd9-2d80-4a02-8629-90ac1400e6ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:09.022503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc59dd9-2d80-4a02-8629-90ac1400e6ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:09.024053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc59dd9-2d80-4a02-8629-90ac1400e6ed None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:09.037915 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 433/864] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:09.043125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ed663-16c6-4468-bbeb-0561c87254fd None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:09.137034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc59dd9-2d80-4a02-8629-90ac1400e6ed None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:09.159479 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ed663-16c6-4468-bbeb-0561c87254fd None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:09.184949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:09.202344 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b827b-862c-4b1f-a09e-0bc400b0b18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:09.203018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b827b-862c-4b1f-a09e-0bc400b0b18e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:09.203296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b827b-862c-4b1f-a09e-0bc400b0b18e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:09.203561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b827b-862c-4b1f-a09e-0bc400b0b18e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:09.211774 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b827b-862c-4b1f-a09e-0bc400b0b18e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:09.213247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b827b-862c-4b1f-a09e-0bc400b0b18e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-776b827b-862c-4b1f-a09e-0bc400b0b18e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:40:09.225854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:09.226785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:09.227681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:40:09.227681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:40:09.227681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:09.227681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:09.227681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:09.227681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:09.227681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:09.227681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:09.227681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 432/865] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:09.232629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b827b-862c-4b1f-a09e-0bc400b0b18e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:09.263640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc59dd9-2d80-4a02-8629-90ac1400e6ed None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:09.266082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:09.309297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc2ba6-c556-4226-8f16-08c431e8120a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:09.310462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc2ba6-c556-4226-8f16-08c431e8120a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:09.310885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc2ba6-c556-4226-8f16-08c431e8120a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:09.311308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc2ba6-c556-4226-8f16-08c431e8120a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:09.312692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc2ba6-c556-4226-8f16-08c431e8120a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:09.314237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc2ba6-c556-4226-8f16-08c431e8120a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-73bc2ba6-c556-4226-8f16-08c431e8120a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:09.331158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:09.332304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:40:09.333405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:09.341243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 434/866] 10.222.0.55 () {56 vars in 1132 bytes} [Thu Sep 8 13:40:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:09.343125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc2ba6-c556-4226-8f16-08c431e8120a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:09.363449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b827b-862c-4b1f-a09e-0bc400b0b18e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:09.402971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2343d35-ed71-4a40-8ece-c45315374482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:09.404268 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2343d35-ed71-4a40-8ece-c45315374482 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:09.404961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2343d35-ed71-4a40-8ece-c45315374482 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:09.405383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2343d35-ed71-4a40-8ece-c45315374482 None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:09.406265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2343d35-ed71-4a40-8ece-c45315374482 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:09.406713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2343d35-ed71-4a40-8ece-c45315374482 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:09.408271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2343d35-ed71-4a40-8ece-c45315374482 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:09.423788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 433/867] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:40:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:09.433190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2343d35-ed71-4a40-8ece-c45315374482 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:09.455073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc2ba6-c556-4226-8f16-08c431e8120a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:09.491614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:09.499997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89dbdf-9e37-427b-a721-987df3c0fa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:09.500743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89dbdf-9e37-427b-a721-987df3c0fa6b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:09.501115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89dbdf-9e37-427b-a721-987df3c0fa6b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:09.502043 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89dbdf-9e37-427b-a721-987df3c0fa6b None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:09.502785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89dbdf-9e37-427b-a721-987df3c0fa6b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:09.503336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89dbdf-9e37-427b-a721-987df3c0fa6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:09.505193 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89dbdf-9e37-427b-a721-987df3c0fa6b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:09.511931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 435/868] 10.222.0.55 () {56 vars in 1126 bytes} [Thu Sep 8 13:40:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:09.517292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89dbdf-9e37-427b-a721-987df3c0fa6b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:09.556572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:09.557205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2343d35-ed71-4a40-8ece-c45315374482 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:09.596038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa55985-e922-4459-a79d-615b27af6894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:09.597020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa55985-e922-4459-a79d-615b27af6894 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:09.597510 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa55985-e922-4459-a79d-615b27af6894 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:09.598036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa55985-e922-4459-a79d-615b27af6894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:09.598951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa55985-e922-4459-a79d-615b27af6894 None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=41a5f117527a41fca00978c35fbd9eb1) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:09.599453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa55985-e922-4459-a79d-615b27af6894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:09.601083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa55985-e922-4459-a79d-615b27af6894 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:09.613953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 434/869] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:09 2022] GET /identity/v3/role_assignments?user.id=41a5f117527a41fca00978c35fbd9eb1&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:09.647699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89dbdf-9e37-427b-a721-987df3c0fa6b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:09.687235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f71aa-f462-4576-9617-4e9112a1f637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:09.687777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f71aa-f462-4576-9617-4e9112a1f637 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:09.688054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f71aa-f462-4576-9617-4e9112a1f637 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:09.688328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f71aa-f462-4576-9617-4e9112a1f637 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:09.688969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f71aa-f462-4576-9617-4e9112a1f637 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:09.689272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f71aa-f462-4576-9617-4e9112a1f637 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:09.690707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f71aa-f462-4576-9617-4e9112a1f637 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:09.696775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 436/870] 10.222.0.55 () {56 vars in 1126 bytes} [Thu Sep 8 13:40:09 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:10.035080 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b3273-10a8-4f6e-bbf0-cea041542f7c None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:10.035596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b3273-10a8-4f6e-bbf0-cea041542f7c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:10.036013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b3273-10a8-4f6e-bbf0-cea041542f7c None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:10.037720 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 435/871] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:10.043564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3cc08-4403-492a-bf81-4f4a601dfa7f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:10.044018 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3cc08-4403-492a-bf81-4f4a601dfa7f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:10.044608 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3cc08-4403-492a-bf81-4f4a601dfa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:10.064178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-41f3cc08-4403-492a-bf81-4f4a601dfa7f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:10.065626 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-41f3cc08-4403-492a-bf81-4f4a601dfa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:10.134820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 437/872] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:10.410939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b3273-10a8-4f6e-bbf0-cea041542f7c None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:10.520537 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b3273-10a8-4f6e-bbf0-cea041542f7c None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:10.530514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:10.557583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e06a94-c9ca-4ecf-9d73-622fc15b6816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:10.558213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e06a94-c9ca-4ecf-9d73-622fc15b6816 None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:10.558489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e06a94-c9ca-4ecf-9d73-622fc15b6816 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:10.558759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e06a94-c9ca-4ecf-9d73-622fc15b6816 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:10.559411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e06a94-c9ca-4ecf-9d73-622fc15b6816 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:10.560767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e06a94-c9ca-4ecf-9d73-622fc15b6816 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:10.577342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 436/873] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:40:10 2022] POST /identity/v3/services => generated 251 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:11.238768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c18f54-1a0a-4e19-8f5d-a3b68bd4ce49 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:11.239282 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c18f54-1a0a-4e19-8f5d-a3b68bd4ce49 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:11.239282 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c18f54-1a0a-4e19-8f5d-a3b68bd4ce49 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:11.240830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 438/874] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:11.246144 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e0aa2-4c4a-48c7-a79a-34e253632da6 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:11.246691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e0aa2-4c4a-48c7-a79a-34e253632da6 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:11.247176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e0aa2-4c4a-48c7-a79a-34e253632da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:11.267172 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-404e0aa2-4c4a-48c7-a79a-34e253632da6 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:11.268689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-404e0aa2-4c4a-48c7-a79a-34e253632da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:11.350542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 437/875] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:11.618010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c18f54-1a0a-4e19-8f5d-a3b68bd4ce49 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:11.644956 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:11.725568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c18f54-1a0a-4e19-8f5d-a3b68bd4ce49 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:11.764204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05827edd-cf22-4209-aae8-52024036c694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:11.764860 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05827edd-cf22-4209-aae8-52024036c694 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:11.765133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05827edd-cf22-4209-aae8-52024036c694 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:11.765465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05827edd-cf22-4209-aae8-52024036c694 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:11.766099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05827edd-cf22-4209-aae8-52024036c694 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:11.767556 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05827edd-cf22-4209-aae8-52024036c694 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-05827edd-cf22-4209-aae8-52024036c694 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:11.777776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:11.778911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:11.779820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 439/876] 10.222.0.55 () {56 vars in 1126 bytes} [Thu Sep 8 13:40:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:11.785366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e0aa2-4c4a-48c7-a79a-34e253632da6 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:11.881296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61303dd7-60bf-46e1-aaf8-a36f8a51f5d5 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:11.882228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61303dd7-60bf-46e1-aaf8-a36f8a51f5d5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:11.882702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61303dd7-60bf-46e1-aaf8-a36f8a51f5d5 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:11.884594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 440/877] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:11.890129 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564772b-71b0-4699-b973-b769b8b4c19d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:11.890706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564772b-71b0-4699-b973-b769b8b4c19d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:11.891346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:11.891731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564772b-71b0-4699-b973-b769b8b4c19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:11.906506 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e0aa2-4c4a-48c7-a79a-34e253632da6 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:11.920261 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f564772b-71b0-4699-b973-b769b8b4c19d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:11.921938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f564772b-71b0-4699-b973-b769b8b4c19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:11.945631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ccc42-1a15-48ca-a217-d2521c9d2dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:11.946622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ccc42-1a15-48ca-a217-d2521c9d2dc5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:11.947117 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ccc42-1a15-48ca-a217-d2521c9d2dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:11.947641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ccc42-1a15-48ca-a217-d2521c9d2dc5 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:11.948484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ccc42-1a15-48ca-a217-d2521c9d2dc5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:11.948996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ccc42-1a15-48ca-a217-d2521c9d2dc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:11.950581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ccc42-1a15-48ca-a217-d2521c9d2dc5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:11.955995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 438/878] 10.222.0.55 () {56 vars in 1122 bytes} [Thu Sep 8 13:40:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:11.962161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ccc42-1a15-48ca-a217-d2521c9d2dc5 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:12.005662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 441/879] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:12.007193 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f8ef1-39f1-4aa5-9f40-bf36d57503f5 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:12.007629 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f8ef1-39f1-4aa5-9f40-bf36d57503f5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:12.008051 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f8ef1-39f1-4aa5-9f40-bf36d57503f5 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:12.009422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 442/880] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:12.014978 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fcf9e-87e7-4be0-8e40-c9803073cbe2 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:12.015267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fcf9e-87e7-4be0-8e40-c9803073cbe2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:12.015544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639fcf9e-87e7-4be0-8e40-c9803073cbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:12.036414 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-639fcf9e-87e7-4be0-8e40-c9803073cbe2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:12.037872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-639fcf9e-87e7-4be0-8e40-c9803073cbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:12.055812 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:12.079642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ccc42-1a15-48ca-a217-d2521c9d2dc5 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:12.120907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d50e7f-a85b-4ea4-8271-86d922455e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:12.121871 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d50e7f-a85b-4ea4-8271-86d922455e92 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:12.122344 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d50e7f-a85b-4ea4-8271-86d922455e92 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:12.122815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d50e7f-a85b-4ea4-8271-86d922455e92 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:12.123624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d50e7f-a85b-4ea4-8271-86d922455e92 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:12.124140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d50e7f-a85b-4ea4-8271-86d922455e92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:12.125562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d50e7f-a85b-4ea4-8271-86d922455e92 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:12.131082 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 439/881] 10.222.0.55 () {56 vars in 1122 bytes} [Thu Sep 8 13:40:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:12.133924 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 443/882] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:12 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:12.300632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d50e7f-a85b-4ea4-8271-86d922455e92 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:12.329248 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:12.410049 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fcf9e-87e7-4be0-8e40-c9803073cbe2 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:12.411937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d50e7f-a85b-4ea4-8271-86d922455e92 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:12.464036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72532c11-df71-423c-99a6-8ec97a616135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:12.464492 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72532c11-df71-423c-99a6-8ec97a616135 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:12.464773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72532c11-df71-423c-99a6-8ec97a616135 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:12.465048 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72532c11-df71-423c-99a6-8ec97a616135 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:12.480199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72532c11-df71-423c-99a6-8ec97a616135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-72532c11-df71-423c-99a6-8ec97a616135 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:12.485385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:12.486394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:12.487265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:12.487265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:12.487265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:12.487265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:12.487265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:12.487265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:12.487265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:12.487265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:12.487265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:12.487265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:12.487265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:12.487265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:12.487938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 440/883] 10.222.0.55 () {56 vars in 1120 bytes} [Thu Sep 8 13:40:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:12.494000 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72532c11-df71-423c-99a6-8ec97a616135 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:12.530564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639fcf9e-87e7-4be0-8e40-c9803073cbe2 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:12.575102 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea49cf-859f-41fe-8b59-07463a64a357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:12.575865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea49cf-859f-41fe-8b59-07463a64a357 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:12.576243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea49cf-859f-41fe-8b59-07463a64a357 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:12.576711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea49cf-859f-41fe-8b59-07463a64a357 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:12.578225 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea49cf-859f-41fe-8b59-07463a64a357 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:12.579671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea49cf-859f-41fe-8b59-07463a64a357 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:12.591269 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:12.607352 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 444/884] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:40:12 2022] POST /identity/v3/services => generated 269 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:40:12.618190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72532c11-df71-423c-99a6-8ec97a616135 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:12.669108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5fa7a-cc1e-44ac-a0a4-e8028b3e14e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:12.669655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5fa7a-cc1e-44ac-a0a4-e8028b3e14e1 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:12.669968 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5fa7a-cc1e-44ac-a0a4-e8028b3e14e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:12.670244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5fa7a-cc1e-44ac-a0a4-e8028b3e14e1 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:12.670952 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5fa7a-cc1e-44ac-a0a4-e8028b3e14e1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:12.671259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5fa7a-cc1e-44ac-a0a4-e8028b3e14e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:12.672835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5fa7a-cc1e-44ac-a0a4-e8028b3e14e1 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:12.684027 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 441/885] 10.222.0.55 () {56 vars in 1115 bytes} [Thu Sep 8 13:40:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:12.690255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea49cf-859f-41fe-8b59-07463a64a357 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:12.704043 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:12.795833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855d2dc-02e6-400a-a7ab-e445452f1a1a None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:12.796690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855d2dc-02e6-400a-a7ab-e445452f1a1a None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:12.796967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9855d2dc-02e6-400a-a7ab-e445452f1a1a None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:12.798561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 442/886] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:12.803600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499a146-ac1f-407d-b45a-07d4ed090828 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:12.803874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499a146-ac1f-407d-b45a-07d4ed090828 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:12.804371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499a146-ac1f-407d-b45a-07d4ed090828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:12.808947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea49cf-859f-41fe-8b59-07463a64a357 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:12.828389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-a499a146-ac1f-407d-b45a-07d4ed090828 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:12.831427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a499a146-ac1f-407d-b45a-07d4ed090828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:12.853802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d189a-76a4-474b-ab79-5a8f3862ec2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:12.855026 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d189a-76a4-474b-ab79-5a8f3862ec2e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:12.855301 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d189a-76a4-474b-ab79-5a8f3862ec2e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:12.855570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d189a-76a4-474b-ab79-5a8f3862ec2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:12.870144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d189a-76a4-474b-ab79-5a8f3862ec2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:12.872452 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d189a-76a4-474b-ab79-5a8f3862ec2e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f92d189a-76a4-474b-ab79-5a8f3862ec2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:12.891061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:12.892335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:12.893420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:12.893420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:12.893420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:12.895175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 445/887] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:12.902735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d189a-76a4-474b-ab79-5a8f3862ec2e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:12.922095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 443/888] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:12.970994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:13.013637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d189a-76a4-474b-ab79-5a8f3862ec2e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.047933 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65741b6-05e0-4852-ab5a-4621d38c6e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.048552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65741b6-05e0-4852-ab5a-4621d38c6e5f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.048832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65741b6-05e0-4852-ab5a-4621d38c6e5f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.049105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65741b6-05e0-4852-ab5a-4621d38c6e5f None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.049771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65741b6-05e0-4852-ab5a-4621d38c6e5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:13.050080 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65741b6-05e0-4852-ab5a-4621d38c6e5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:13.051441 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65741b6-05e0-4852-ab5a-4621d38c6e5f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:13.064720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 446/889] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:13.070456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499a146-ac1f-407d-b45a-07d4ed090828 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.145670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:13.183893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499a146-ac1f-407d-b45a-07d4ed090828 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.203009 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65741b6-05e0-4852-ab5a-4621d38c6e5f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.220592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251aaa1-721f-42b9-a5f4-69095b08d367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.221563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251aaa1-721f-42b9-a5f4-69095b08d367 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.222142 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251aaa1-721f-42b9-a5f4-69095b08d367 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.222630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251aaa1-721f-42b9-a5f4-69095b08d367 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.230652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251aaa1-721f-42b9-a5f4-69095b08d367 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:13.232389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251aaa1-721f-42b9-a5f4-69095b08d367 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-7251aaa1-721f-42b9-a5f4-69095b08d367 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:40:13.244376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:13.245327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:13.246351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:13.246351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:13.246351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:40:13.246351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:40:13.246351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:40:13.246351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:40:13.246351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:40:13.246351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:40:13.246351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:40:13.246351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:13.253292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 444/890] 10.222.0.55 () {56 vars in 1117 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:13.254365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251aaa1-721f-42b9-a5f4-69095b08d367 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.304969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65741b6-05e0-4852-ab5a-4621d38c6e5f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.326260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:13.342764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011bd89-e7da-457a-9b81-40e07029d46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.343745 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011bd89-e7da-457a-9b81-40e07029d46f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.344238 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011bd89-e7da-457a-9b81-40e07029d46f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.344715 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011bd89-e7da-457a-9b81-40e07029d46f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.345513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011bd89-e7da-457a-9b81-40e07029d46f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:13.347144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011bd89-e7da-457a-9b81-40e07029d46f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-3011bd89-e7da-457a-9b81-40e07029d46f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:13.356895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:13.358177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:40:13.359036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:13.359786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251aaa1-721f-42b9-a5f4-69095b08d367 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.361675 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 447/891] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:13.369908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011bd89-e7da-457a-9b81-40e07029d46f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.392702 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:13.398342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72c018-4acf-4e25-ac26-a2af39c3f5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.398969 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72c018-4acf-4e25-ac26-a2af39c3f5ea None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.399248 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72c018-4acf-4e25-ac26-a2af39c3f5ea None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.399518 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72c018-4acf-4e25-ac26-a2af39c3f5ea None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.400319 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72c018-4acf-4e25-ac26-a2af39c3f5ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:13.400631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72c018-4acf-4e25-ac26-a2af39c3f5ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:13.402609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72c018-4acf-4e25-ac26-a2af39c3f5ea None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:13.428627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 445/892] 10.222.0.55 () {56 vars in 1149 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:13.429593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72c018-4acf-4e25-ac26-a2af39c3f5ea None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.479083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011bd89-e7da-457a-9b81-40e07029d46f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.516136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24225454-3fca-4808-91e8-6a79ee23f5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.517124 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24225454-3fca-4808-91e8-6a79ee23f5fe None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.517675 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24225454-3fca-4808-91e8-6a79ee23f5fe None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.518305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24225454-3fca-4808-91e8-6a79ee23f5fe None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.519137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24225454-3fca-4808-91e8-6a79ee23f5fe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:13.519642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24225454-3fca-4808-91e8-6a79ee23f5fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:13.521303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24225454-3fca-4808-91e8-6a79ee23f5fe None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:13.527344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 448/893] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:13.533954 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24225454-3fca-4808-91e8-6a79ee23f5fe None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.543237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72c018-4acf-4e25-ac26-a2af39c3f5ea None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.582508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29076d2e-c8d3-40ae-8e0a-1fa236226e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.583288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076d2e-c8d3-40ae-8e0a-1fa236226e3d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.583571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076d2e-c8d3-40ae-8e0a-1fa236226e3d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.583896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076d2e-c8d3-40ae-8e0a-1fa236226e3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.587893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:13.641765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24225454-3fca-4808-91e8-6a79ee23f5fe None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.647031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:13.662778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29076d2e-c8d3-40ae-8e0a-1fa236226e3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:13.664226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29076d2e-c8d3-40ae-8e0a-1fa236226e3d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:13.678109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f501c7-c589-4f7e-8c20-764bd56d8be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.678985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f501c7-c589-4f7e-8c20-764bd56d8be1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.679493 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f501c7-c589-4f7e-8c20-764bd56d8be1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.679980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f501c7-c589-4f7e-8c20-764bd56d8be1 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.680804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f501c7-c589-4f7e-8c20-764bd56d8be1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:13.681321 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f501c7-c589-4f7e-8c20-764bd56d8be1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:13.682945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f501c7-c589-4f7e-8c20-764bd56d8be1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:13.688484 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 449/894] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:13.695540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f501c7-c589-4f7e-8c20-764bd56d8be1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.793149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 446/895] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:40:13.794463 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29076d2e-c8d3-40ae-8e0a-1fa236226e3d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.812092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f501c7-c589-4f7e-8c20-764bd56d8be1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.825463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:13.847909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b6e11-b9cd-4c3b-89e6-c006c3eaf23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.848536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b6e11-b9cd-4c3b-89e6-c006c3eaf23b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.848813 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b6e11-b9cd-4c3b-89e6-c006c3eaf23b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.849147 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b6e11-b9cd-4c3b-89e6-c006c3eaf23b None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.849992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b6e11-b9cd-4c3b-89e6-c006c3eaf23b None admin] RBAC: Adding query filter params (interface=public, service_id=f55f67f8a73f4fba8e9f0ce236078acc, region_id=RegionOne) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:13.850219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b6e11-b9cd-4c3b-89e6-c006c3eaf23b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:13.851595 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b6e11-b9cd-4c3b-89e6-c006c3eaf23b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:13.862066 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 450/896] 10.222.0.55 () {56 vars in 1251 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/endpoints?service_id=f55f67f8a73f4fba8e9f0ce236078acc&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:13.863333 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b6e11-b9cd-4c3b-89e6-c006c3eaf23b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.920184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:13.939942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29076d2e-c8d3-40ae-8e0a-1fa236226e3d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.987936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4403d0-9388-457c-a8c6-40ce92bddd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.989061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4403d0-9388-457c-a8c6-40ce92bddd16 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.989653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4403d0-9388-457c-a8c6-40ce92bddd16 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.990184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4403d0-9388-457c-a8c6-40ce92bddd16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.001119 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b6e11-b9cd-4c3b-89e6-c006c3eaf23b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:14.005368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4403d0-9388-457c-a8c6-40ce92bddd16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:14.007154 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4403d0-9388-457c-a8c6-40ce92bddd16 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-4d4403d0-9388-457c-a8c6-40ce92bddd16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:14.027780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:14.028916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:14.029865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:14.029865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:14.029865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:14.036833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 447/897] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:14.037215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4403d0-9388-457c-a8c6-40ce92bddd16 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:14.048094 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb750530-0da5-4e90-8076-e670b77c0c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:14.048634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb750530-0da5-4e90-8076-e670b77c0c95 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:14.048971 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb750530-0da5-4e90-8076-e670b77c0c95 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:14.049261 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb750530-0da5-4e90-8076-e670b77c0c95 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.049953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb750530-0da5-4e90-8076-e670b77c0c95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:14.051330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb750530-0da5-4e90-8076-e670b77c0c95 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:14.057767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 451/898] 10.222.0.55 () {56 vars in 1087 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/services => generated 1268 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:40:14.158944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4403d0-9388-457c-a8c6-40ce92bddd16 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:14.201365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed984f8-8c0b-44eb-8d7a-72def459d59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:14.202015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed984f8-8c0b-44eb-8d7a-72def459d59e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:14.202294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed984f8-8c0b-44eb-8d7a-72def459d59e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:14.202567 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed984f8-8c0b-44eb-8d7a-72def459d59e None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.203233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed984f8-8c0b-44eb-8d7a-72def459d59e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:14.203538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed984f8-8c0b-44eb-8d7a-72def459d59e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:14.204924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed984f8-8c0b-44eb-8d7a-72def459d59e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:14.216266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:14.225450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 448/899] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:14.226676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb750530-0da5-4e90-8076-e670b77c0c95 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:14.276739 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:14.346348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb750530-0da5-4e90-8076-e670b77c0c95 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:14.383164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c092f-15e0-43e2-b604-9e044149c334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:14.384100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c092f-15e0-43e2-b604-9e044149c334 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:14.384562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c092f-15e0-43e2-b604-9e044149c334 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:14.385084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c092f-15e0-43e2-b604-9e044149c334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.392443 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c092f-15e0-43e2-b604-9e044149c334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:14.394147 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c092f-15e0-43e2-b604-9e044149c334 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-407c092f-15e0-43e2-b604-9e044149c334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:40:14.406776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:14.407791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:14.408676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:40:14.408676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:40:14.408676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:14.408676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:14.408676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:14.408676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:14.408676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:14.408676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:14.410798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 452/900] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:14.415476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a975c-9712-40d9-9bfd-7c7862984b30 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:14.415476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a975c-9712-40d9-9bfd-7c7862984b30 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:14.415476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a975c-9712-40d9-9bfd-7c7862984b30 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.415779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 449/901] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:14.417401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c092f-15e0-43e2-b604-9e044149c334 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:14.420764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaebe07-ed9b-4d6b-9a45-2c19b111e87f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:14.422550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaebe07-ed9b-4d6b-9a45-2c19b111e87f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:14.422829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaebe07-ed9b-4d6b-9a45-2c19b111e87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.441356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1eaebe07-ed9b-4d6b-9a45-2c19b111e87f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:14.442698 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1eaebe07-ed9b-4d6b-9a45-2c19b111e87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:14.526868 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:14.529778 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c092f-15e0-43e2-b604-9e044149c334 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:14.545845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 450/902] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:14.565453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3dfa3-5116-4561-bb93-f516137a1c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:14.566161 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3dfa3-5116-4561-bb93-f516137a1c44 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:14.566439 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3dfa3-5116-4561-bb93-f516137a1c44 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:14.566763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3dfa3-5116-4561-bb93-f516137a1c44 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.567511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3dfa3-5116-4561-bb93-f516137a1c44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:14.567818 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3dfa3-5116-4561-bb93-f516137a1c44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:14.569545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3dfa3-5116-4561-bb93-f516137a1c44 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:14.582842 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 453/903] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:40:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:14.588899 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaebe07-ed9b-4d6b-9a45-2c19b111e87f None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:14.696970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaebe07-ed9b-4d6b-9a45-2c19b111e87f None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:14.726620 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:14.735012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4c9f66-3a9a-4fcd-bad9-d8fddabbe95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:14.735624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c9f66-3a9a-4fcd-bad9-d8fddabbe95d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:14.735901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c9f66-3a9a-4fcd-bad9-d8fddabbe95d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:14.736172 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4c9f66-3a9a-4fcd-bad9-d8fddabbe95d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.736901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c9f66-3a9a-4fcd-bad9-d8fddabbe95d None admin] RBAC: Adding query filter params (role.id=a50724aa8b014431acb0d653bf1e41e7, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=41a5f117527a41fca00978c35fbd9eb1) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:14.737206 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c9f66-3a9a-4fcd-bad9-d8fddabbe95d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:14.738663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4c9f66-3a9a-4fcd-bad9-d8fddabbe95d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:14.744832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 451/904] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:14 2022] GET /identity/v3/role_assignments?user.id=41a5f117527a41fca00978c35fbd9eb1&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=a50724aa8b014431acb0d653bf1e41e7 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:14.850406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3dfa3-5116-4561-bb93-f516137a1c44 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:14.889312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cd62d-0b76-4436-aef1-2f39acdeaaba None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:14.889635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cd62d-0b76-4436-aef1-2f39acdeaaba None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:14.889896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cd62d-0b76-4436-aef1-2f39acdeaaba None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.891403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 452/905] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:14.896945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05318e2d-d6bc-47fb-9f97-302a53e4df04 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:14.897260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05318e2d-d6bc-47fb-9f97-302a53e4df04 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:14.897495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05318e2d-d6bc-47fb-9f97-302a53e4df04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:14.918226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-05318e2d-d6bc-47fb-9f97-302a53e4df04 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:14.919493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-05318e2d-d6bc-47fb-9f97-302a53e4df04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:14.975526 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3dfa3-5116-4561-bb93-f516137a1c44 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:14.996493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 453/906] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:15.012650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e2d83-8e7d-4400-8418-5ecd718b6207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:15.013514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e2d83-8e7d-4400-8418-5ecd718b6207 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:15.013970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e2d83-8e7d-4400-8418-5ecd718b6207 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:15.014413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e2d83-8e7d-4400-8418-5ecd718b6207 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:15.015265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e2d83-8e7d-4400-8418-5ecd718b6207 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:15.016823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e2d83-8e7d-4400-8418-5ecd718b6207 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:15.034478 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 454/907] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:40:14 2022] POST /identity/v3/services => generated 241 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:15.263180 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05318e2d-d6bc-47fb-9f97-302a53e4df04 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:15.363548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05318e2d-d6bc-47fb-9f97-302a53e4df04 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:15.386684 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:15.403425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6810a6-d19d-45bf-9939-f7f44e039c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:15.404057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6810a6-d19d-45bf-9939-f7f44e039c79 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:15.404328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6810a6-d19d-45bf-9939-f7f44e039c79 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:15.404594 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6810a6-d19d-45bf-9939-f7f44e039c79 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:15.405174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6810a6-d19d-45bf-9939-f7f44e039c79 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:15.406553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6810a6-d19d-45bf-9939-f7f44e039c79 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-bc6810a6-d19d-45bf-9939-f7f44e039c79 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:15.415544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:15.417295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:15.418213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 454/908] 10.222.0.55 () {56 vars in 1132 bytes} [Thu Sep 8 13:40:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:15.423565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e2d83-8e7d-4400-8418-5ecd718b6207 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:15.447371 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:15.534767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e2d83-8e7d-4400-8418-5ecd718b6207 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:15.571702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888c6e5-4249-46a7-83e1-8c826b3dcb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:15.572862 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888c6e5-4249-46a7-83e1-8c826b3dcb53 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:15.573140 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888c6e5-4249-46a7-83e1-8c826b3dcb53 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:15.573411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888c6e5-4249-46a7-83e1-8c826b3dcb53 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:15.574160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888c6e5-4249-46a7-83e1-8c826b3dcb53 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:15.574468 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888c6e5-4249-46a7-83e1-8c826b3dcb53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:15.575831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888c6e5-4249-46a7-83e1-8c826b3dcb53 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:15.580967 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 455/909] 10.222.0.55 () {56 vars in 1126 bytes} [Thu Sep 8 13:40:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:15.586215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6810a6-d19d-45bf-9939-f7f44e039c79 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:15.693104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6810a6-d19d-45bf-9939-f7f44e039c79 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:15.727307 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4920e-3ee1-41b7-bf7e-7edfcaf31fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:15.727917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4920e-3ee1-41b7-bf7e-7edfcaf31fa8 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:15.728194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4920e-3ee1-41b7-bf7e-7edfcaf31fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:15.729003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4920e-3ee1-41b7-bf7e-7edfcaf31fa8 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:15.729691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4920e-3ee1-41b7-bf7e-7edfcaf31fa8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:15.730015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4920e-3ee1-41b7-bf7e-7edfcaf31fa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:15.731364 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb4920e-3ee1-41b7-bf7e-7edfcaf31fa8 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:15.736631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 455/910] 10.222.0.55 () {56 vars in 1126 bytes} [Thu Sep 8 13:40:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:15.742829 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888c6e5-4249-46a7-83e1-8c826b3dcb53 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:15.830177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:15.847033 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888c6e5-4249-46a7-83e1-8c826b3dcb53 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:15.882260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d14e41-31d1-4317-a22f-5ff5dcf4101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:15.882865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d14e41-31d1-4317-a22f-5ff5dcf4101c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:15.883137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d14e41-31d1-4317-a22f-5ff5dcf4101c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:15.883406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d14e41-31d1-4317-a22f-5ff5dcf4101c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:15.884097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d14e41-31d1-4317-a22f-5ff5dcf4101c None admin] RBAC: Adding query filter params (interface=public, service_id=a41d4035cd4f422dbeeb2bd597409b98, region_id=RegionOne) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:15.884397 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d14e41-31d1-4317-a22f-5ff5dcf4101c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:15.885801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d14e41-31d1-4317-a22f-5ff5dcf4101c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:15.891226 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 456/911] 10.222.0.55 () {56 vars in 1251 bytes} [Thu Sep 8 13:40:15 2022] GET /identity/v3/endpoints?service_id=a41d4035cd4f422dbeeb2bd597409b98&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:15.897317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4920e-3ee1-41b7-bf7e-7edfcaf31fa8 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:15.947754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:16.001921 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4920e-3ee1-41b7-bf7e-7edfcaf31fa8 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:16.036756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeeb6e6-f267-4105-a8f4-73049f72b89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:16.037365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeeb6e6-f267-4105-a8f4-73049f72b89c None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:16.037644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeeb6e6-f267-4105-a8f4-73049f72b89c None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:16.037943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeeb6e6-f267-4105-a8f4-73049f72b89c None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:16.038613 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeeb6e6-f267-4105-a8f4-73049f72b89c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:16.040025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeeb6e6-f267-4105-a8f4-73049f72b89c None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:16.046615 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 456/912] 10.222.0.55 () {56 vars in 1087 bytes} [Thu Sep 8 13:40:15 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:40:16.429573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cd64b-7642-4107-a838-11cdc1b29b04 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:16.429945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cd64b-7642-4107-a838-11cdc1b29b04 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:16.430187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cd64b-7642-4107-a838-11cdc1b29b04 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:16.432380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 457/913] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:16.437741 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884fd95-f44d-412d-af88-dd552af1a13c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:16.438103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884fd95-f44d-412d-af88-dd552af1a13c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:16.438377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884fd95-f44d-412d-af88-dd552af1a13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:16.457123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-e884fd95-f44d-412d-af88-dd552af1a13c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:16.458326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e884fd95-f44d-412d-af88-dd552af1a13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:16.532613 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 457/914] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:16.794772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cd64b-7642-4107-a838-11cdc1b29b04 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:16.908983 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cd64b-7642-4107-a838-11cdc1b29b04 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:16.945535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63020f-7e05-42cf-94bd-5c45bb885178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:16.946200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63020f-7e05-42cf-94bd-5c45bb885178 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:16.946475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63020f-7e05-42cf-94bd-5c45bb885178 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:16.946744 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63020f-7e05-42cf-94bd-5c45bb885178 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:16.947400 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63020f-7e05-42cf-94bd-5c45bb885178 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:16.948762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63020f-7e05-42cf-94bd-5c45bb885178 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-4e63020f-7e05-42cf-94bd-5c45bb885178 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:16.958200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:16.959208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:16.960052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 458/915] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:40:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:16.967106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884fd95-f44d-412d-af88-dd552af1a13c None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:16.975947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49648147-efb8-43ed-8711-d600954d52b4 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:16.976228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49648147-efb8-43ed-8711-d600954d52b4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:16.976501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49648147-efb8-43ed-8711-d600954d52b4 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:16.978289 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 459/916] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:16.983716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6ee07-b7f9-4977-820d-b226014164b8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:16.984095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6ee07-b7f9-4977-820d-b226014164b8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:16.984376 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6ee07-b7f9-4977-820d-b226014164b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.006346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-57c6ee07-b7f9-4977-820d-b226014164b8 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:17.007536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-57c6ee07-b7f9-4977-820d-b226014164b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:17.014605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:17.030099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:17.094221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 460/917] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:17.099978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884fd95-f44d-412d-af88-dd552af1a13c None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:17.136380 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec66cc9-4a5d-410b-a8ff-bd5bc5aa1899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:17.137313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec66cc9-4a5d-410b-a8ff-bd5bc5aa1899 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:17.137825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec66cc9-4a5d-410b-a8ff-bd5bc5aa1899 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:17.138295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec66cc9-4a5d-410b-a8ff-bd5bc5aa1899 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.139183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec66cc9-4a5d-410b-a8ff-bd5bc5aa1899 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:17.139744 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec66cc9-4a5d-410b-a8ff-bd5bc5aa1899 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:17.141310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec66cc9-4a5d-410b-a8ff-bd5bc5aa1899 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:17.151428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 458/918] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:40:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:17.153086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6ee07-b7f9-4977-820d-b226014164b8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:17.268416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6ee07-b7f9-4977-820d-b226014164b8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:17.303293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a0a91f-8263-495e-a8a8-69b43afc9237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:17.303970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0a91f-8263-495e-a8a8-69b43afc9237 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:17.304254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0a91f-8263-495e-a8a8-69b43afc9237 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:17.304522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a0a91f-8263-495e-a8a8-69b43afc9237 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.305152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0a91f-8263-495e-a8a8-69b43afc9237 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:17.305496 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0a91f-8263-495e-a8a8-69b43afc9237 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:17.307008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a0a91f-8263-495e-a8a8-69b43afc9237 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:17.312272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 461/919] 10.222.0.55 () {56 vars in 1114 bytes} [Thu Sep 8 13:40:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:17.317994 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec66cc9-4a5d-410b-a8ff-bd5bc5aa1899 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:17.336707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ee173-01ef-4a4b-ae77-570dcc8c8445 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:17.336707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ee173-01ef-4a4b-ae77-570dcc8c8445 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:17.336707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ee173-01ef-4a4b-ae77-570dcc8c8445 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.336707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 462/920] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:17.341389 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8798775-c852-41da-a2ff-d24bee65c8c8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:17.342308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8798775-c852-41da-a2ff-d24bee65c8c8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:17.342819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8798775-c852-41da-a2ff-d24bee65c8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.359385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:17.367040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f8798775-c852-41da-a2ff-d24bee65c8c8 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:17.368562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f8798775-c852-41da-a2ff-d24bee65c8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:17.445194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec66cc9-4a5d-410b-a8ff-bd5bc5aa1899 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:17.451840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 463/921] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:17.453360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:17.454612 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8798775-c852-41da-a2ff-d24bee65c8c8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:17.490534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78ab2b-9540-46ec-8f1a-e01d59769f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:17.491169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78ab2b-9540-46ec-8f1a-e01d59769f90 None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:17.491450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78ab2b-9540-46ec-8f1a-e01d59769f90 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:17.491720 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78ab2b-9540-46ec-8f1a-e01d59769f90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.492547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78ab2b-9540-46ec-8f1a-e01d59769f90 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:17.494394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78ab2b-9540-46ec-8f1a-e01d59769f90 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:17.526352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 459/922] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:40:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:17.575988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8798775-c852-41da-a2ff-d24bee65c8c8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:17.616689 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:17.622502 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11971f-1834-49f1-bdbf-00b2a37e02e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:17.623578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11971f-1834-49f1-bdbf-00b2a37e02e6 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:17.624057 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11971f-1834-49f1-bdbf-00b2a37e02e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:17.624527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11971f-1834-49f1-bdbf-00b2a37e02e6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.637057 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11971f-1834-49f1-bdbf-00b2a37e02e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1f11971f-1834-49f1-bdbf-00b2a37e02e6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:17.642385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:17.643343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:17.644247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:17.644247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:17.644247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:17.644247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:17.644247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:17.644247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:17.644247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:17.644247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:17.644247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:17.644247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:17.644247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:17.644247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:17.647089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 464/923] 10.222.0.55 () {56 vars in 1120 bytes} [Thu Sep 8 13:40:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:17.652994 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78ab2b-9540-46ec-8f1a-e01d59769f90 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:17.722589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11971f-1834-49f1-bdbf-00b2a37e02e6 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:17.770364 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78ab2b-9540-46ec-8f1a-e01d59769f90 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:17.812824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d3a5f-db1f-41b8-88a6-a2e6cc6e233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:17.813449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d3a5f-db1f-41b8-88a6-a2e6cc6e233d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:17.813727 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d3a5f-db1f-41b8-88a6-a2e6cc6e233d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:17.814081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d3a5f-db1f-41b8-88a6-a2e6cc6e233d None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.814753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d3a5f-db1f-41b8-88a6-a2e6cc6e233d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:17.815522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d3a5f-db1f-41b8-88a6-a2e6cc6e233d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:17.816893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d3a5f-db1f-41b8-88a6-a2e6cc6e233d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:17.829541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 460/924] 10.222.0.55 () {56 vars in 1115 bytes} [Thu Sep 8 13:40:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:17.835796 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d3a5f-db1f-41b8-88a6-a2e6cc6e233d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:17.845480 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11971f-1834-49f1-bdbf-00b2a37e02e6 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:17.869861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:17.885798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2599e8-86a1-4cc2-9a4c-76d95a9da0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:17.886565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2599e8-86a1-4cc2-9a4c-76d95a9da0f0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:17.887038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2599e8-86a1-4cc2-9a4c-76d95a9da0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:17.887376 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2599e8-86a1-4cc2-9a4c-76d95a9da0f0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:17.888098 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2599e8-86a1-4cc2-9a4c-76d95a9da0f0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:17.889547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2599e8-86a1-4cc2-9a4c-76d95a9da0f0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-8d2599e8-86a1-4cc2-9a4c-76d95a9da0f0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:17.903447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:17.904514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:17.905351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:17.909639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 465/925] 10.222.0.55 () {56 vars in 1126 bytes} [Thu Sep 8 13:40:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:17.918808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2599e8-86a1-4cc2-9a4c-76d95a9da0f0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:17.970330 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d3a5f-db1f-41b8-88a6-a2e6cc6e233d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:17.990590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:18.013688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7688344-00a3-47a8-9744-c83e64025de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:18.014478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7688344-00a3-47a8-9744-c83e64025de0 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:18.014853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7688344-00a3-47a8-9744-c83e64025de0 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:18.015664 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7688344-00a3-47a8-9744-c83e64025de0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:18.029592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7688344-00a3-47a8-9744-c83e64025de0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:18.031727 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7688344-00a3-47a8-9744-c83e64025de0 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:18.045548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2599e8-86a1-4cc2-9a4c-76d95a9da0f0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-b7688344-00a3-47a8-9744-c83e64025de0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:18.049839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:18.050773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:18.051668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:18.051668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:18.051668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:18.051668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 461/926] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:18.058887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7688344-00a3-47a8-9744-c83e64025de0 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:18.098420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a61c15-2f19-44f1-ad92-3c61dbea4a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:18.099332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a61c15-2f19-44f1-ad92-3c61dbea4a15 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:18.099609 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a61c15-2f19-44f1-ad92-3c61dbea4a15 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:18.099883 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a61c15-2f19-44f1-ad92-3c61dbea4a15 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:18.100541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a61c15-2f19-44f1-ad92-3c61dbea4a15 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:18.100850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a61c15-2f19-44f1-ad92-3c61dbea4a15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:18.103721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a61c15-2f19-44f1-ad92-3c61dbea4a15 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:18.110350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 466/927] 10.222.0.55 () {56 vars in 1122 bytes} [Thu Sep 8 13:40:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:18.117142 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a61c15-2f19-44f1-ad92-3c61dbea4a15 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:18.164264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:18.193052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7688344-00a3-47a8-9744-c83e64025de0 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:18.231581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33bb152-3171-44bd-ae0d-15e4432dd1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:18.231950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a61c15-2f19-44f1-ad92-3c61dbea4a15 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:18.232185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bb152-3171-44bd-ae0d-15e4432dd1dd None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:18.232463 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bb152-3171-44bd-ae0d-15e4432dd1dd None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:18.232734 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33bb152-3171-44bd-ae0d-15e4432dd1dd None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:18.233376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bb152-3171-44bd-ae0d-15e4432dd1dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:18.233676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bb152-3171-44bd-ae0d-15e4432dd1dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:18.235063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33bb152-3171-44bd-ae0d-15e4432dd1dd None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:18.252140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 462/928] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:18.252140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1837ee-d6be-4b78-9886-73c1dea5c8ec None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:18.252140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1837ee-d6be-4b78-9886-73c1dea5c8ec None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:18.252140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1837ee-d6be-4b78-9886-73c1dea5c8ec None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:18.252140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 463/929] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:18.255498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1837ee-d6be-4b78-9886-73c1dea5c8ec None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:18.257494 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:18.273048 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306a6359-0102-49ec-b68a-bc215790ed92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:18.273654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a6359-0102-49ec-b68a-bc215790ed92 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:18.273956 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a6359-0102-49ec-b68a-bc215790ed92 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:18.274226 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306a6359-0102-49ec-b68a-bc215790ed92 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:18.274853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a6359-0102-49ec-b68a-bc215790ed92 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:18.275181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a6359-0102-49ec-b68a-bc215790ed92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:18.276524 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306a6359-0102-49ec-b68a-bc215790ed92 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:18.285961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 467/930] 10.222.0.55 () {56 vars in 1122 bytes} [Thu Sep 8 13:40:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:18.285961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb3799-d960-4354-808c-06d576487f8f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:18.285961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb3799-d960-4354-808c-06d576487f8f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:18.285961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb3799-d960-4354-808c-06d576487f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:18.300359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0beb3799-d960-4354-808c-06d576487f8f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:18.301511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0beb3799-d960-4354-808c-06d576487f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:18.371665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 468/931] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:18.372634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb3799-d960-4354-808c-06d576487f8f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:18.375476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1837ee-d6be-4b78-9886-73c1dea5c8ec None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:18.418442 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9748bc-8472-455c-906b-75662f23c829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:18.418983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9748bc-8472-455c-906b-75662f23c829 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:18.419262 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9748bc-8472-455c-906b-75662f23c829 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:18.419534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9748bc-8472-455c-906b-75662f23c829 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:18.423203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:18.429615 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9748bc-8472-455c-906b-75662f23c829 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:18.431223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9748bc-8472-455c-906b-75662f23c829 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-9d9748bc-8472-455c-906b-75662f23c829 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:40:18.451860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:18.452831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:18.453827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:18.453827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:18.453827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:40:18.453827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:40:18.453827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:40:18.453827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:40:18.453827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:40:18.453827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:40:18.453827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:40:18.453827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:18.459774 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 464/932] 10.222.0.55 () {56 vars in 1117 bytes} [Thu Sep 8 13:40:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:18.460441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9748bc-8472-455c-906b-75662f23c829 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:18.519456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb3799-d960-4354-808c-06d576487f8f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:18.535630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:18.558523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60893f-fd08-44c1-8503-a1a6a4a39477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:18.559469 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60893f-fd08-44c1-8503-a1a6a4a39477 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:18.559961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60893f-fd08-44c1-8503-a1a6a4a39477 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:18.560507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60893f-fd08-44c1-8503-a1a6a4a39477 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:18.561554 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60893f-fd08-44c1-8503-a1a6a4a39477 None admin] RBAC: Adding query filter params (interface=public, service_id=0f3fe7d18e8e4bb399c9d29df184e4e2, region_id=RegionOne) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:18.562117 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60893f-fd08-44c1-8503-a1a6a4a39477 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:18.563860 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60893f-fd08-44c1-8503-a1a6a4a39477 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:18.569568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 469/933] 10.222.0.55 () {56 vars in 1251 bytes} [Thu Sep 8 13:40:18 2022] GET /identity/v3/endpoints?service_id=0f3fe7d18e8e4bb399c9d29df184e4e2&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:18.576446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60893f-fd08-44c1-8503-a1a6a4a39477 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:18.586864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9748bc-8472-455c-906b-75662f23c829 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:18.626819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a224fe-2c7d-4e08-89a0-584dce9bbe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:18.627498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a224fe-2c7d-4e08-89a0-584dce9bbe38 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:18.627778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a224fe-2c7d-4e08-89a0-584dce9bbe38 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:18.628108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a224fe-2c7d-4e08-89a0-584dce9bbe38 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:18.629275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a224fe-2c7d-4e08-89a0-584dce9bbe38 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:18.629585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a224fe-2c7d-4e08-89a0-584dce9bbe38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:18.631483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a224fe-2c7d-4e08-89a0-584dce9bbe38 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:18.656768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 465/934] 10.222.0.55 () {56 vars in 1149 bytes} [Thu Sep 8 13:40:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:18.657652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a224fe-2c7d-4e08-89a0-584dce9bbe38 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:18.699497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60893f-fd08-44c1-8503-a1a6a4a39477 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:18.715867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:18.736966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6baa0-9b9f-474a-ac13-28138f512121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:18.738081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6baa0-9b9f-474a-ac13-28138f512121 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:18.738563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6baa0-9b9f-474a-ac13-28138f512121 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:18.739185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6baa0-9b9f-474a-ac13-28138f512121 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:18.739797 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6baa0-9b9f-474a-ac13-28138f512121 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:18.741201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6baa0-9b9f-474a-ac13-28138f512121 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:18.758103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 470/935] 10.222.0.55 () {56 vars in 1087 bytes} [Thu Sep 8 13:40:18 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:40:18.759000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6baa0-9b9f-474a-ac13-28138f512121 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:18.763015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a224fe-2c7d-4e08-89a0-584dce9bbe38 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:18.826917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb40d0-d64e-4949-9cac-018f7b383d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:18.826917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb40d0-d64e-4949-9cac-018f7b383d8e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:18.826917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb40d0-d64e-4949-9cac-018f7b383d8e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:18.826917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb40d0-d64e-4949-9cac-018f7b383d8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:18.842674 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb40d0-d64e-4949-9cac-018f7b383d8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:18.845227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb40d0-d64e-4949-9cac-018f7b383d8e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:18.847866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-7ddb40d0-d64e-4949-9cac-018f7b383d8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:18.873942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:18.874986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:18.875887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:18.875887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:18.875887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:18.878744 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 466/936] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:18.885752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb40d0-d64e-4949-9cac-018f7b383d8e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:18.928992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6baa0-9b9f-474a-ac13-28138f512121 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:18.980589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ffeb9-6b81-4fc5-9f94-051256cfd628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:18.981163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ffeb9-6b81-4fc5-9f94-051256cfd628 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:18.981437 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ffeb9-6b81-4fc5-9f94-051256cfd628 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:18.981706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ffeb9-6b81-4fc5-9f94-051256cfd628 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:18.982328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ffeb9-6b81-4fc5-9f94-051256cfd628 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:18.984331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ffeb9-6b81-4fc5-9f94-051256cfd628 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-402ffeb9-6b81-4fc5-9f94-051256cfd628 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:18.997686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:18.999753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:19.001005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 471/937] 10.222.0.55 () {56 vars in 1132 bytes} [Thu Sep 8 13:40:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:19.004881 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ffeb9-6b81-4fc5-9f94-051256cfd628 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:19.036686 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb40d0-d64e-4949-9cac-018f7b383d8e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:19.070120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:19.096167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23e857-8a9f-4958-9ca8-8f8ab692f928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:19.096784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23e857-8a9f-4958-9ca8-8f8ab692f928 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.097065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23e857-8a9f-4958-9ca8-8f8ab692f928 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.097336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23e857-8a9f-4958-9ca8-8f8ab692f928 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.098271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23e857-8a9f-4958-9ca8-8f8ab692f928 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:19.098573 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23e857-8a9f-4958-9ca8-8f8ab692f928 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:19.100703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23e857-8a9f-4958-9ca8-8f8ab692f928 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:19.119549 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 467/938] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:19.120653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23e857-8a9f-4958-9ca8-8f8ab692f928 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:19.122086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ffeb9-6b81-4fc5-9f94-051256cfd628 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:19.130937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:19.165067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b212c-7b6b-4120-b8ef-7afbb8770158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:19.166171 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b212c-7b6b-4120-b8ef-7afbb8770158 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.167406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b212c-7b6b-4120-b8ef-7afbb8770158 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.167864 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b212c-7b6b-4120-b8ef-7afbb8770158 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.168693 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b212c-7b6b-4120-b8ef-7afbb8770158 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:19.169169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b212c-7b6b-4120-b8ef-7afbb8770158 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:19.170737 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b212c-7b6b-4120-b8ef-7afbb8770158 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:19.182364 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 472/939] 10.222.0.55 () {56 vars in 1126 bytes} [Thu Sep 8 13:40:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:19.187320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b212c-7b6b-4120-b8ef-7afbb8770158 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:19.255458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23e857-8a9f-4958-9ca8-8f8ab692f928 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:19.304387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf94fbe-8e86-4cef-bad8-efb168869237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:19.305247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf94fbe-8e86-4cef-bad8-efb168869237 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.305550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf94fbe-8e86-4cef-bad8-efb168869237 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.305857 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf94fbe-8e86-4cef-bad8-efb168869237 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.316031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf94fbe-8e86-4cef-bad8-efb168869237 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:19.316667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b212c-7b6b-4120-b8ef-7afbb8770158 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:19.317480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf94fbe-8e86-4cef-bad8-efb168869237 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-3cf94fbe-8e86-4cef-bad8-efb168869237 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:40:19.333668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:19.334677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:19.335591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:40:19.335591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:40:19.335591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:19.335591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:19.335591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:19.335591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:19.335591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:19.335591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:19.337808 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 468/940] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:19.341390 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf94fbe-8e86-4cef-bad8-efb168869237 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:19.360115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f92a3d-7ad5-42a3-a4ef-ccda51181dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:19.361180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f92a3d-7ad5-42a3-a4ef-ccda51181dd0 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.361699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f92a3d-7ad5-42a3-a4ef-ccda51181dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.362314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f92a3d-7ad5-42a3-a4ef-ccda51181dd0 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.363278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f92a3d-7ad5-42a3-a4ef-ccda51181dd0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:19.363795 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f92a3d-7ad5-42a3-a4ef-ccda51181dd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:19.365363 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f92a3d-7ad5-42a3-a4ef-ccda51181dd0 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:19.376634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 473/941] 10.222.0.55 () {56 vars in 1126 bytes} [Thu Sep 8 13:40:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:19.377920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f92a3d-7ad5-42a3-a4ef-ccda51181dd0 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:19.378396 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:19.425279 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:19.470540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf94fbe-8e86-4cef-bad8-efb168869237 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:19.512067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f92a3d-7ad5-42a3-a4ef-ccda51181dd0 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:19.517258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fee8e-a324-43b2-b5b7-f81516cf8c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:19.517908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fee8e-a324-43b2-b5b7-f81516cf8c10 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.518188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fee8e-a324-43b2-b5b7-f81516cf8c10 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.518460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fee8e-a324-43b2-b5b7-f81516cf8c10 None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.519159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fee8e-a324-43b2-b5b7-f81516cf8c10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:19.519465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fee8e-a324-43b2-b5b7-f81516cf8c10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:19.520895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fee8e-a324-43b2-b5b7-f81516cf8c10 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:19.541905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 469/942] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:40:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:19.545242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fee8e-a324-43b2-b5b7-f81516cf8c10 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:19.555208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061645e-0c7c-4245-87cb-032b400a61e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:19.556155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061645e-0c7c-4245-87cb-032b400a61e3 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.556752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061645e-0c7c-4245-87cb-032b400a61e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.557234 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061645e-0c7c-4245-87cb-032b400a61e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.558216 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061645e-0c7c-4245-87cb-032b400a61e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:19.559915 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061645e-0c7c-4245-87cb-032b400a61e3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:19.585224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 474/943] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:40:19 2022] POST /identity/v3/endpoints => generated 347 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:19.658776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:19.678733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fee8e-a324-43b2-b5b7-f81516cf8c10 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:19.730313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8691fd-de02-4816-8394-93218f54a369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:19.731020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8691fd-de02-4816-8394-93218f54a369 None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.731296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8691fd-de02-4816-8394-93218f54a369 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.731568 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8691fd-de02-4816-8394-93218f54a369 None admin] PATH_INFO: `/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/41a5f117527a41fca00978c35fbd9eb1/roles/a50724aa8b014431acb0d653bf1e41e7` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.734189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8691fd-de02-4816-8394-93218f54a369 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe9c4e7bc14e70b57bf834cfff365f, user_id=41a5f117527a41fca00978c35fbd9eb1, role_id=a50724aa8b014431acb0d653bf1e41e7)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:19.735800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8691fd-de02-4816-8394-93218f54a369 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:19.757965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 470/944] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:40:19 2022] PUT /identity/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/41a5f117527a41fca00978c35fbd9eb1/roles/a50724aa8b014431acb0d653bf1e41e7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:40:19.846986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480028e1-1586-40ae-9cc6-758764f67312 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.847288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480028e1-1586-40ae-9cc6-758764f67312 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.847516 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480028e1-1586-40ae-9cc6-758764f67312 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.849297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 475/945] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:19.854859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaabfe3-d291-4555-87f7-e9f2c6a7b8d3 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.855163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaabfe3-d291-4555-87f7-e9f2c6a7b8d3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.855411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaabfe3-d291-4555-87f7-e9f2c6a7b8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.876458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-dbaabfe3-d291-4555-87f7-e9f2c6a7b8d3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:19.877758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-dbaabfe3-d291-4555-87f7-e9f2c6a7b8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:19.954901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 471/946] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:20.221674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480028e1-1586-40ae-9cc6-758764f67312 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:20.317491 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:20.332201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480028e1-1586-40ae-9cc6-758764f67312 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:20.367790 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a7999-2d75-4489-bb43-7ac4ab056009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:20.368396 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a7999-2d75-4489-bb43-7ac4ab056009 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:20.369287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a7999-2d75-4489-bb43-7ac4ab056009 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:20.369572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a7999-2d75-4489-bb43-7ac4ab056009 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:20.370242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a7999-2d75-4489-bb43-7ac4ab056009 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:20.371666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a7999-2d75-4489-bb43-7ac4ab056009 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:20.374336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: WARNING keystone.server.flask.common [None req-dd5a7999-2d75-4489-bb43-7ac4ab056009 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 08 13:40:20.393499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.sql.core [None req-dd5a7999-2d75-4489-bb43-7ac4ab056009 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 13:40:20.393499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 13:40:20.393499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [parameters: {'id': 'a4d6f89b5b404ec2820ba2b70e6166ec', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 13:40:20.393499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87916) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 13:40:20.410256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-dd5a7999-2d75-4489-bb43-7ac4ab056009 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a4d6f89b5b404ec2820ba2b70e6166ec.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a4d6f89b5b404ec2820ba2b70e6166ec. Sep 08 13:40:20.410256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:20.410256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:40:20.410256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:40:20.410256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:40:20.410256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:40:20.410256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:40:20.410256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:40:20.410256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:40:20.410256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:40:20.410256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:40:20.410256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:40:20.410256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result.read() Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 08 13:40:20.411092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application yield resource Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise self.value Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application session.commit() Sep 08 13:40:20.412054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.session.flush() Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application compat.raise_( Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 13:40:20.412939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 13:40:20.413872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application result.read() Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:40:20.414651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [parameters: {'id': 'a4d6f89b5b404ec2820ba2b70e6166ec', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:20.415574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:20.416980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:20.421877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:20.421877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 08 13:40:20.421877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 08 13:40:20.421877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:20.421877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:20.421877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 08 13:40:20.421877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 08 13:40:20.421877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a4d6f89b5b404ec2820ba2b70e6166ec. Sep 08 13:40:20.421877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:20.421877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 476/947] 10.222.0.55 () {60 vars in 1140 bytes} [Thu Sep 8 13:40:20 2022] POST /identity/v3/projects => generated 262 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:20.422566 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaabfe3-d291-4555-87f7-e9f2c6a7b8d3 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:20.508331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:20.542132 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaabfe3-d291-4555-87f7-e9f2c6a7b8d3 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:20.577199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b5b77-7036-4a1a-850d-4fe75bdd245b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:20.577855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b5b77-7036-4a1a-850d-4fe75bdd245b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:20.578140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b5b77-7036-4a1a-850d-4fe75bdd245b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:20.578411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b5b77-7036-4a1a-850d-4fe75bdd245b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:20.585356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b5b77-7036-4a1a-850d-4fe75bdd245b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:20.586823 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b5b77-7036-4a1a-850d-4fe75bdd245b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-ce6b5b77-7036-4a1a-850d-4fe75bdd245b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:40:20.599025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:20.600898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:20.603005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:40:20.603005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:40:20.603005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:20.603005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:20.603005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:20.603005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:20.603005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:20.603005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:20.603005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 472/948] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:20.607397 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a7999-2d75-4489-bb43-7ac4ab056009 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:20.715234 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a7999-2d75-4489-bb43-7ac4ab056009 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:20.753567 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ee705-3707-44b8-8ec2-bf13e7a7a26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:20.754212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ee705-3707-44b8-8ec2-bf13e7a7a26a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:20.754492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ee705-3707-44b8-8ec2-bf13e7a7a26a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:20.754761 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ee705-3707-44b8-8ec2-bf13e7a7a26a None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:20.755429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ee705-3707-44b8-8ec2-bf13e7a7a26a None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:20.755730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ee705-3707-44b8-8ec2-bf13e7a7a26a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:20.757361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ee705-3707-44b8-8ec2-bf13e7a7a26a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:20.786119 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 477/949] 10.222.0.55 () {56 vars in 1112 bytes} [Thu Sep 8 13:40:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:21.094640 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a4a2d-cedb-4953-8b1b-af7df31fa632 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:21.100922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a4a2d-cedb-4953-8b1b-af7df31fa632 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:21.100922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a4a2d-cedb-4953-8b1b-af7df31fa632 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:21.101496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 473/950] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:21.107513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e5e30-770b-4100-93d8-5f1f39852a88 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:21.108388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e5e30-770b-4100-93d8-5f1f39852a88 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:21.108942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e5e30-770b-4100-93d8-5f1f39852a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:21.124247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:21.130654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c95e5e30-770b-4100-93d8-5f1f39852a88 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:21.132530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c95e5e30-770b-4100-93d8-5f1f39852a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:21.202295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 478/951] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:21.479308 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a4a2d-cedb-4953-8b1b-af7df31fa632 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:21.586967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a4a2d-cedb-4953-8b1b-af7df31fa632 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:21.621121 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:21.623458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b2cad2-ad3c-4d93-a8bd-d59eada3962c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:21.624099 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2cad2-ad3c-4d93-a8bd-d59eada3962c None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:21.624379 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2cad2-ad3c-4d93-a8bd-d59eada3962c None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:21.624663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2cad2-ad3c-4d93-a8bd-d59eada3962c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:21.625254 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b2cad2-ad3c-4d93-a8bd-d59eada3962c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:21.626833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b2cad2-ad3c-4d93-a8bd-d59eada3962c None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-08b2cad2-ad3c-4d93-a8bd-d59eada3962c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:21.636779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:21.637827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:21.638882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 474/952] 10.222.0.55 () {56 vars in 1126 bytes} [Thu Sep 8 13:40:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:21.646121 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e5e30-770b-4100-93d8-5f1f39852a88 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:21.751997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e5e30-770b-4100-93d8-5f1f39852a88 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:21.788720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc179c43-eb67-4763-b520-f3dc6fa94cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:21.789339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc179c43-eb67-4763-b520-f3dc6fa94cef None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:21.789616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc179c43-eb67-4763-b520-f3dc6fa94cef None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:21.789919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc179c43-eb67-4763-b520-f3dc6fa94cef None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:21.791118 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc179c43-eb67-4763-b520-f3dc6fa94cef None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:21.791421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc179c43-eb67-4763-b520-f3dc6fa94cef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:21.793020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc179c43-eb67-4763-b520-f3dc6fa94cef None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:21.795034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:21.799500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277736b-a62d-4693-a277-44b28554299b None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:21.799700 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 479/953] 10.222.0.55 () {56 vars in 1122 bytes} [Thu Sep 8 13:40:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:21.799838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277736b-a62d-4693-a277-44b28554299b None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:21.800046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277736b-a62d-4693-a277-44b28554299b None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:21.801484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 475/954] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:21.806840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277736b-a62d-4693-a277-44b28554299b None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:21.807330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c283d8b-4446-4a59-abcd-a7a3079ee4d7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:21.807632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c283d8b-4446-4a59-abcd-a7a3079ee4d7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:21.807890 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c283d8b-4446-4a59-abcd-a7a3079ee4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:21.828469 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8c283d8b-4446-4a59-abcd-a7a3079ee4d7 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:21.829654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8c283d8b-4446-4a59-abcd-a7a3079ee4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:21.915885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 480/955] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:21.935901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277736b-a62d-4693-a277-44b28554299b None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:21.974219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52a91c-85c4-4938-9705-ab4d76611fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:21.974824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52a91c-85c4-4938-9705-ab4d76611fc3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:21.975094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52a91c-85c4-4938-9705-ab4d76611fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:21.975364 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52a91c-85c4-4938-9705-ab4d76611fc3 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:21.975997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52a91c-85c4-4938-9705-ab4d76611fc3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:21.976293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52a91c-85c4-4938-9705-ab4d76611fc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:21.977654 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52a91c-85c4-4938-9705-ab4d76611fc3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:21.983012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 476/956] 10.222.0.55 () {56 vars in 1122 bytes} [Thu Sep 8 13:40:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:21.988602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c283d8b-4446-4a59-abcd-a7a3079ee4d7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:22.052691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80406f1-fd27-413c-ae1a-df88ab551b8e None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:22.052993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80406f1-fd27-413c-ae1a-df88ab551b8e None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:22.053191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80406f1-fd27-413c-ae1a-df88ab551b8e None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:22.054783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 477/957] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:22.059597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c5c4b-10c2-4905-bd0c-7152f9ba6635 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:22.059882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c5c4b-10c2-4905-bd0c-7152f9ba6635 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:22.060155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9c5c4b-10c2-4905-bd0c-7152f9ba6635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:22.078747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-cd9c5c4b-10c2-4905-bd0c-7152f9ba6635 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:22.079940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-cd9c5c4b-10c2-4905-bd0c-7152f9ba6635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:22.111772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c283d8b-4446-4a59-abcd-a7a3079ee4d7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:22.143048 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:22.152265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:22.159564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 478/958] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:22.159853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb40238-b17e-4513-8040-805629cb1056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:22.160473 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb40238-b17e-4513-8040-805629cb1056 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:22.160755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb40238-b17e-4513-8040-805629cb1056 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:22.161030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb40238-b17e-4513-8040-805629cb1056 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:22.166100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb40238-b17e-4513-8040-805629cb1056 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:22.166100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb40238-b17e-4513-8040-805629cb1056 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:22.185463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 481/959] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:40:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:22.188607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c5c4b-10c2-4905-bd0c-7152f9ba6635 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:22.305602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9c5c4b-10c2-4905-bd0c-7152f9ba6635 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:22.352472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8fc65-a380-4223-a379-e196ad6ddece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:22.353647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8fc65-a380-4223-a379-e196ad6ddece None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:22.353959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8fc65-a380-4223-a379-e196ad6ddece None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:22.354233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8fc65-a380-4223-a379-e196ad6ddece None admin] PATH_INFO: `/v3/services/compute` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:22.354818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8fc65-a380-4223-a379-e196ad6ddece None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:22.356188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8fc65-a380-4223-a379-e196ad6ddece None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-65a8fc65-a380-4223-a379-e196ad6ddece None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:22.365553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:22.366660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:22.367656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 479/960] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:22.372954 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb40238-b17e-4513-8040-805629cb1056 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:22.440036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8fc65-a380-4223-a379-e196ad6ddece None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:22.495933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:22.507399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb40238-b17e-4513-8040-805629cb1056 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:22.551208 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8fc65-a380-4223-a379-e196ad6ddece None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:22.555010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2149a6-286b-4ec9-8b22-603058f98722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:22.555970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2149a6-286b-4ec9-8b22-603058f98722 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:22.556493 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2149a6-286b-4ec9-8b22-603058f98722 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:22.556973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2149a6-286b-4ec9-8b22-603058f98722 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:22.557840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2149a6-286b-4ec9-8b22-603058f98722 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:22.558395 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2149a6-286b-4ec9-8b22-603058f98722 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:22.560585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2149a6-286b-4ec9-8b22-603058f98722 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:22.573764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 482/961] 10.222.0.55 () {56 vars in 1112 bytes} [Thu Sep 8 13:40:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:22.577039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2149a6-286b-4ec9-8b22-603058f98722 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:22.592132 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed13ba-0c14-4c0c-85dc-48a4e37329eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:22.592942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed13ba-0c14-4c0c-85dc-48a4e37329eb None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:22.593403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed13ba-0c14-4c0c-85dc-48a4e37329eb None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:22.593756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed13ba-0c14-4c0c-85dc-48a4e37329eb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:22.605812 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed13ba-0c14-4c0c-85dc-48a4e37329eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-d7ed13ba-0c14-4c0c-85dc-48a4e37329eb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:22.611987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:22.612959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:22.613852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:22.613852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:22.613852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:22.613852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:22.613852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:22.613852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:22.613852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:22.613852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:22.613852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:22.613852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:22.613852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:22.613852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:22.613852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 480/962] 10.222.0.55 () {56 vars in 1120 bytes} [Thu Sep 8 13:40:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:22.620325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed13ba-0c14-4c0c-85dc-48a4e37329eb None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:22.637256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:22.704627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2149a6-286b-4ec9-8b22-603058f98722 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:22.739421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed13ba-0c14-4c0c-85dc-48a4e37329eb None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:22.747141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc03b5a-df9a-4ff3-9f79-d4d49e3bf9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:22.747761 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc03b5a-df9a-4ff3-9f79-d4d49e3bf9a5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:22.748040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc03b5a-df9a-4ff3-9f79-d4d49e3bf9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:22.748316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc03b5a-df9a-4ff3-9f79-d4d49e3bf9a5 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:22.748952 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc03b5a-df9a-4ff3-9f79-d4d49e3bf9a5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:22.749256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc03b5a-df9a-4ff3-9f79-d4d49e3bf9a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:22.750680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc03b5a-df9a-4ff3-9f79-d4d49e3bf9a5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:22.754147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:22.761762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 483/963] 10.222.0.55 () {56 vars in 1112 bytes} [Thu Sep 8 13:40:22 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:22.784256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe07d4-6a4e-4ed1-afc5-058e9c04ced6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:22.784884 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe07d4-6a4e-4ed1-afc5-058e9c04ced6 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:22.785221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe07d4-6a4e-4ed1-afc5-058e9c04ced6 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:22.785497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe07d4-6a4e-4ed1-afc5-058e9c04ced6 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:22.786275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe07d4-6a4e-4ed1-afc5-058e9c04ced6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:22.786582 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe07d4-6a4e-4ed1-afc5-058e9c04ced6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:22.788049 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe07d4-6a4e-4ed1-afc5-058e9c04ced6 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:22.803951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 481/964] 10.222.0.55 () {56 vars in 1115 bytes} [Thu Sep 8 13:40:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:22.805528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc03b5a-df9a-4ff3-9f79-d4d49e3bf9a5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:22.933907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc03b5a-df9a-4ff3-9f79-d4d49e3bf9a5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:22.956162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:22.980999 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab0dc9-b653-4814-93af-7032568fe5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:22.981644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab0dc9-b653-4814-93af-7032568fe5c2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:22.981976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab0dc9-b653-4814-93af-7032568fe5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:22.982250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab0dc9-b653-4814-93af-7032568fe5c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:22.993801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab0dc9-b653-4814-93af-7032568fe5c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:22.995233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab0dc9-b653-4814-93af-7032568fe5c2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-38ab0dc9-b653-4814-93af-7032568fe5c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:23.013273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:23.014417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:23.015340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:23.015340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:23.015340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:23.019488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 484/965] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:23.021144 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe07d4-6a4e-4ed1-afc5-058e9c04ced6 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:23.132119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe07d4-6a4e-4ed1-afc5-058e9c04ced6 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:23.135701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbd67b-4a8d-4672-9d1f-9bd582544f29 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:23.136133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbd67b-4a8d-4672-9d1f-9bd582544f29 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:23.136230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bbd67b-4a8d-4672-9d1f-9bd582544f29 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:23.137857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 485/966] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:23.142894 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4d106-8623-4ab5-9e21-96b5e38fff2b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:23.143174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4d106-8623-4ab5-9e21-96b5e38fff2b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:23.143445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4d106-8623-4ab5-9e21-96b5e38fff2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:23.163719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-00d4d106-8623-4ab5-9e21-96b5e38fff2b None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:23.164994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-00d4d106-8623-4ab5-9e21-96b5e38fff2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:23.167776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a70273-c890-45ce-8c18-858226f9eb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:23.168406 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a70273-c890-45ce-8c18-858226f9eb7f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:23.168687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a70273-c890-45ce-8c18-858226f9eb7f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:23.168962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a70273-c890-45ce-8c18-858226f9eb7f None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:23.169634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a70273-c890-45ce-8c18-858226f9eb7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:23.169977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a70273-c890-45ce-8c18-858226f9eb7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:23.171415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a70273-c890-45ce-8c18-858226f9eb7f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:23.185977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 482/967] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:23.194714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a70273-c890-45ce-8c18-858226f9eb7f None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:23.231521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:23.257985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 486/968] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:23.307753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a70273-c890-45ce-8c18-858226f9eb7f None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:23.343766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452b2b7-5ddb-4a8d-9f63-c82f2b8f0768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:23.344392 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452b2b7-5ddb-4a8d-9f63-c82f2b8f0768 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:23.344670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452b2b7-5ddb-4a8d-9f63-c82f2b8f0768 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:23.344936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452b2b7-5ddb-4a8d-9f63-c82f2b8f0768 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:23.352398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452b2b7-5ddb-4a8d-9f63-c82f2b8f0768 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:23.354437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452b2b7-5ddb-4a8d-9f63-c82f2b8f0768 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-a452b2b7-5ddb-4a8d-9f63-c82f2b8f0768 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:40:23.366092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:23.367232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:23.368228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:23.368228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:23.368228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:40:23.368228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:40:23.368228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:40:23.368228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:40:23.368228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:40:23.368228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:40:23.368228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:40:23.368228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:23.372026 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 483/969] 10.222.0.55 () {56 vars in 1117 bytes} [Thu Sep 8 13:40:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:23.373376 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4d106-8623-4ab5-9e21-96b5e38fff2b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:23.474967 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:23.482362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4d106-8623-4ab5-9e21-96b5e38fff2b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:23.517192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc62afb-26c9-448e-808c-4ea129ee6e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:23.518246 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc62afb-26c9-448e-808c-4ea129ee6e11 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:23.518729 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc62afb-26c9-448e-808c-4ea129ee6e11 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:23.519186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc62afb-26c9-448e-808c-4ea129ee6e11 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:23.520131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc62afb-26c9-448e-808c-4ea129ee6e11 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:23.520634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc62afb-26c9-448e-808c-4ea129ee6e11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:23.522292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc62afb-26c9-448e-808c-4ea129ee6e11 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:23.542750 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 487/970] 10.222.0.55 () {56 vars in 1149 bytes} [Thu Sep 8 13:40:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:23.547310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452b2b7-5ddb-4a8d-9f63-c82f2b8f0768 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:23.550691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc62afb-26c9-448e-808c-4ea129ee6e11 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:23.655916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:23.666996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452b2b7-5ddb-4a8d-9f63-c82f2b8f0768 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:23.688568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc62afb-26c9-448e-808c-4ea129ee6e11 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:23.705232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1bad3b-5f5a-4f23-8ac2-360c108f4073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:23.706161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1bad3b-5f5a-4f23-8ac2-360c108f4073 None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:23.706533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1bad3b-5f5a-4f23-8ac2-360c108f4073 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:23.707180 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1bad3b-5f5a-4f23-8ac2-360c108f4073 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:23.708090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1bad3b-5f5a-4f23-8ac2-360c108f4073 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:23.709512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1bad3b-5f5a-4f23-8ac2-360c108f4073 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:23.712083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: WARNING keystone.server.flask.common [None req-fa1bad3b-5f5a-4f23-8ac2-360c108f4073 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 08 13:40:23.726642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.sql.core [None req-fa1bad3b-5f5a-4f23-8ac2-360c108f4073 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 13:40:23.726642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 13:40:23.726642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [parameters: {'user_id': '18bf1bdc606b45e58447e7feed5cd2be', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 13:40:23.726642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87917) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 13:40:23.730941 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc69d4-fdbb-4715-9458-1158140777ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:23.731653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc69d4-fdbb-4715-9458-1158140777ed None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:23.732040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc69d4-fdbb-4715-9458-1158140777ed None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:23.732380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc69d4-fdbb-4715-9458-1158140777ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-fa1bad3b-5f5a-4f23-8ac2-360c108f4073 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application result.read() Sep 08 13:40:23.735988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:40:23.736971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application yield resource Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise self.value Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application session.commit() Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 13:40:23.738003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self.session.flush() Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application compat.raise_( Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 13:40:23.738927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:40:23.739945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application result.read() Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:40:23.740821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [parameters: {'user_id': '18bf1bdc606b45e58447e7feed5cd2be', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:23.741673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 08 13:40:23.744524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 08 13:40:23.755709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 08 13:40:23.755709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 08 13:40:23.755709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 13:40:23.755709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 13:40:23.755709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 13:40:23.755709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:23.755709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 484/971] 10.222.0.55 () {60 vars in 1131 bytes} [Thu Sep 8 13:40:23 2022] POST /identity/v3/users => generated 160 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:23.755709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1bad3b-5f5a-4f23-8ac2-360c108f4073 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:23.764099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:23.827282 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc69d4-fdbb-4715-9458-1158140777ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:23.830001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc69d4-fdbb-4715-9458-1158140777ed None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:23.860881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1bad3b-5f5a-4f23-8ac2-360c108f4073 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:23.902678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f9117-fac0-42d3-8129-3e83f88cb70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:23.903315 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f9117-fac0-42d3-8129-3e83f88cb70b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:23.903594 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f9117-fac0-42d3-8129-3e83f88cb70b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:23.903865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027f9117-fac0-42d3-8129-3e83f88cb70b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:23.917415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f9117-fac0-42d3-8129-3e83f88cb70b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:23.919357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027f9117-fac0-42d3-8129-3e83f88cb70b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-027f9117-fac0-42d3-8129-3e83f88cb70b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:40:23.935103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:23.936095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:23.937050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:23.937050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:23.937050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:40:23.937050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:40:23.937050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:40:23.937050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:40:23.937050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:40:23.937050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:40:23.937050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:40:23.937050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:23.941452 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 485/972] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:23.942852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f9117-fac0-42d3-8129-3e83f88cb70b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:23.966533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:23.967899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 488/973] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:23 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:40:23.976047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc69d4-fdbb-4715-9458-1158140777ed None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:24.049854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:24.080197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027f9117-fac0-42d3-8129-3e83f88cb70b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:24.097692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc69d4-fdbb-4715-9458-1158140777ed None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:24.126076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa622620-1e57-47b3-b6cb-6c11706883a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:24.126961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa622620-1e57-47b3-b6cb-6c11706883a5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:24.127242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa622620-1e57-47b3-b6cb-6c11706883a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:24.127515 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa622620-1e57-47b3-b6cb-6c11706883a5 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:24.128376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa622620-1e57-47b3-b6cb-6c11706883a5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:24.128685 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa622620-1e57-47b3-b6cb-6c11706883a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:24.130777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa622620-1e57-47b3-b6cb-6c11706883a5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:24.134050 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253a525-edd9-4d1f-aaa1-440f183de507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:24.134843 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253a525-edd9-4d1f-aaa1-440f183de507 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:24.135223 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253a525-edd9-4d1f-aaa1-440f183de507 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:24.135594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253a525-edd9-4d1f-aaa1-440f183de507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:24.146234 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253a525-edd9-4d1f-aaa1-440f183de507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:24.147938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253a525-edd9-4d1f-aaa1-440f183de507 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-8253a525-edd9-4d1f-aaa1-440f183de507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:24.168390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:24.169916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:24.171769 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:24.171769 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:24.171769 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:24.174594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 489/974] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:24.175660 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253a525-edd9-4d1f-aaa1-440f183de507 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:24.176130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 486/975] 10.222.0.55 () {56 vars in 1101 bytes} [Thu Sep 8 13:40:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:24.290298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253a525-edd9-4d1f-aaa1-440f183de507 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:24.298761 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:24.329420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a8f91-ca05-4f30-8c48-7767231dfed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:24.329997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a8f91-ca05-4f30-8c48-7767231dfed8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:24.330276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a8f91-ca05-4f30-8c48-7767231dfed8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:24.330541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a8f91-ca05-4f30-8c48-7767231dfed8 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:24.331194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a8f91-ca05-4f30-8c48-7767231dfed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:24.331495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a8f91-ca05-4f30-8c48-7767231dfed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:24.332871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a8f91-ca05-4f30-8c48-7767231dfed8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:24.352540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 490/976] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:24.354612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa622620-1e57-47b3-b6cb-6c11706883a5 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:24.434433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:24.469003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa622620-1e57-47b3-b6cb-6c11706883a5 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:24.474492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff56f5-5961-4b7a-88f7-595177609fe7 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:24.474785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff56f5-5961-4b7a-88f7-595177609fe7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:24.475059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff56f5-5961-4b7a-88f7-595177609fe7 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:24.476534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 491/977] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:24.481861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e81cff-97a2-40ac-ad6c-686bbbd1fa32 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:24.482181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e81cff-97a2-40ac-ad6c-686bbbd1fa32 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:24.482423 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e81cff-97a2-40ac-ad6c-686bbbd1fa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:24.500010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-19e81cff-97a2-40ac-ad6c-686bbbd1fa32 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:24.501187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-19e81cff-97a2-40ac-ad6c-686bbbd1fa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:24.507531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f0e5a-4f7e-45c5-a96c-e4ee5db0782b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:24.508155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f0e5a-4f7e-45c5-a96c-e4ee5db0782b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:24.508843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f0e5a-4f7e-45c5-a96c-e4ee5db0782b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:24.509651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f0e5a-4f7e-45c5-a96c-e4ee5db0782b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:24.518488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f0e5a-4f7e-45c5-a96c-e4ee5db0782b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:24.520233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f0e5a-4f7e-45c5-a96c-e4ee5db0782b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-939f0e5a-4f7e-45c5-a96c-e4ee5db0782b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:40:24.533724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:24.534743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:24.540737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:40:24.540737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:40:24.540737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:24.540737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:24.540737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:24.540737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:24.540737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:24.540737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:24.540737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 487/978] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:24.543378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f0e5a-4f7e-45c5-a96c-e4ee5db0782b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:24.574785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 492/979] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:24.646943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f0e5a-4f7e-45c5-a96c-e4ee5db0782b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:24.676922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:24.683449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4ee70-7fb3-4fb5-b926-dd698916d713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:24.684090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4ee70-7fb3-4fb5-b926-dd698916d713 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:24.684453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4ee70-7fb3-4fb5-b926-dd698916d713 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:24.684759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f4ee70-7fb3-4fb5-b926-dd698916d713 None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:24.685457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4ee70-7fb3-4fb5-b926-dd698916d713 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:24.685786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4ee70-7fb3-4fb5-b926-dd698916d713 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:24.687208 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f4ee70-7fb3-4fb5-b926-dd698916d713 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:24.700656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 488/980] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:40:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:24.706550 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e81cff-97a2-40ac-ad6c-686bbbd1fa32 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:24.811433 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:24.828441 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e81cff-97a2-40ac-ad6c-686bbbd1fa32 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:24.849929 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4ee70-7fb3-4fb5-b926-dd698916d713 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:24.873989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944380b-df9c-4ccb-9641-315497d31c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:24.874953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944380b-df9c-4ccb-9641-315497d31c8f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:24.875318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944380b-df9c-4ccb-9641-315497d31c8f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:24.875806 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944380b-df9c-4ccb-9641-315497d31c8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:24.877172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944380b-df9c-4ccb-9641-315497d31c8f None admin] RBAC: Adding query filter params (role.id=a50724aa8b014431acb0d653bf1e41e7, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=41a5f117527a41fca00978c35fbd9eb1) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:24.877775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944380b-df9c-4ccb-9641-315497d31c8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:24.880493 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944380b-df9c-4ccb-9641-315497d31c8f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:24.901846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 493/981] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:24 2022] GET /identity/v3/role_assignments?user.id=41a5f117527a41fca00978c35fbd9eb1&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=a50724aa8b014431acb0d653bf1e41e7 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:24.992676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f4ee70-7fb3-4fb5-b926-dd698916d713 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:25.030185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08013969-24e2-4f87-8b3d-01262774ed74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:25.030833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08013969-24e2-4f87-8b3d-01262774ed74 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:25.031114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08013969-24e2-4f87-8b3d-01262774ed74 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:25.031387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08013969-24e2-4f87-8b3d-01262774ed74 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:25.032001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08013969-24e2-4f87-8b3d-01262774ed74 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:25.033409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08013969-24e2-4f87-8b3d-01262774ed74 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-08013969-24e2-4f87-8b3d-01262774ed74 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:25.043289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:25.044404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:25.046643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 489/982] 10.222.0.55 () {56 vars in 1126 bytes} [Thu Sep 8 13:40:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:25.051486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944380b-df9c-4ccb-9641-315497d31c8f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:25.071383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f012694-2f1a-4fa4-9263-59bc07960a90 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:25.071789 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f012694-2f1a-4fa4-9263-59bc07960a90 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:25.072570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f012694-2f1a-4fa4-9263-59bc07960a90 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:25.074118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 490/983] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:25.079168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747d62d-00cd-48c2-a850-89f883d90af6 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:25.079490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747d62d-00cd-48c2-a850-89f883d90af6 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:25.079721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747d62d-00cd-48c2-a850-89f883d90af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:25.099607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-5747d62d-00cd-48c2-a850-89f883d90af6 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:25.101236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5747d62d-00cd-48c2-a850-89f883d90af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:25.158130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:25.166772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944380b-df9c-4ccb-9641-315497d31c8f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:25.175638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 491/984] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:25.205216 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bcc0f-2523-458b-8647-29a72a78226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:25.205852 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bcc0f-2523-458b-8647-29a72a78226d None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:25.206135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bcc0f-2523-458b-8647-29a72a78226d None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:25.206405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404bcc0f-2523-458b-8647-29a72a78226d None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:25.207029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bcc0f-2523-458b-8647-29a72a78226d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:25.207332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bcc0f-2523-458b-8647-29a72a78226d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:25.208721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404bcc0f-2523-458b-8647-29a72a78226d None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:25.218693 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 494/985] 10.222.0.55 () {56 vars in 1122 bytes} [Thu Sep 8 13:40:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:25.219869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747d62d-00cd-48c2-a850-89f883d90af6 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:25.322892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747d62d-00cd-48c2-a850-89f883d90af6 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:25.356223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a679505-5f02-4b2c-89a4-2711cde58cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:25.356821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a679505-5f02-4b2c-89a4-2711cde58cae None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:25.357096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a679505-5f02-4b2c-89a4-2711cde58cae None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:25.357362 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a679505-5f02-4b2c-89a4-2711cde58cae None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:25.358010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a679505-5f02-4b2c-89a4-2711cde58cae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:25.358308 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a679505-5f02-4b2c-89a4-2711cde58cae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:25.359657 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a679505-5f02-4b2c-89a4-2711cde58cae None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:25.364636 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 492/986] 10.222.0.55 () {56 vars in 1122 bytes} [Thu Sep 8 13:40:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:25.370315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bcc0f-2523-458b-8647-29a72a78226d None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:25.395717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:25.455460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a679505-5f02-4b2c-89a4-2711cde58cae None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:25.474629 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404bcc0f-2523-458b-8647-29a72a78226d None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:25.508097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:25.509675 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034714fc-f55f-47a5-bf65-1e38cb51aa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:25.510372 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034714fc-f55f-47a5-bf65-1e38cb51aa51 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:25.510678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034714fc-f55f-47a5-bf65-1e38cb51aa51 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:25.510947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034714fc-f55f-47a5-bf65-1e38cb51aa51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:25.511919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034714fc-f55f-47a5-bf65-1e38cb51aa51 None admin] RBAC: Adding query filter params (interface=admin, service_id=0f3fe7d18e8e4bb399c9d29df184e4e2, region_id=RegionOne) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:25.512224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034714fc-f55f-47a5-bf65-1e38cb51aa51 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:25.513574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034714fc-f55f-47a5-bf65-1e38cb51aa51 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:25.518840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 495/987] 10.222.0.55 () {56 vars in 1249 bytes} [Thu Sep 8 13:40:25 2022] GET /identity/v3/endpoints?service_id=0f3fe7d18e8e4bb399c9d29df184e4e2&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:25.524593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034714fc-f55f-47a5-bf65-1e38cb51aa51 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:25.563508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a679505-5f02-4b2c-89a4-2711cde58cae None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:25.602253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b2269-e5a9-4485-9953-0ac743d02fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:25.603455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b2269-e5a9-4485-9953-0ac743d02fe6 None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:25.603743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b2269-e5a9-4485-9953-0ac743d02fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:25.604024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b2269-e5a9-4485-9953-0ac743d02fe6 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:25.604683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b2269-e5a9-4485-9953-0ac743d02fe6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:25.606072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b2269-e5a9-4485-9953-0ac743d02fe6 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:25.618556 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:25.621237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 493/988] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:40:25 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:40:25.636774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034714fc-f55f-47a5-bf65-1e38cb51aa51 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:25.684178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca80f8c-0295-4b96-b90f-188a2f04ce6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:25.684813 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca80f8c-0295-4b96-b90f-188a2f04ce6f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:25.685089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca80f8c-0295-4b96-b90f-188a2f04ce6f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:25.685358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca80f8c-0295-4b96-b90f-188a2f04ce6f None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:25.685996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca80f8c-0295-4b96-b90f-188a2f04ce6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:25.687373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca80f8c-0295-4b96-b90f-188a2f04ce6f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:25.701777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 496/989] 10.222.0.55 () {56 vars in 1087 bytes} [Thu Sep 8 13:40:25 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:40:27.112347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3add3b4-931c-4275-8992-81095eef5aa9 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:27.112856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3add3b4-931c-4275-8992-81095eef5aa9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:27.112856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3add3b4-931c-4275-8992-81095eef5aa9 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:27.114291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 494/990] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:27.119905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26580c-76f5-478e-a273-d632f26be21a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:27.120337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26580c-76f5-478e-a273-d632f26be21a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:27.120513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c26580c-76f5-478e-a273-d632f26be21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:27.139589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9c26580c-76f5-478e-a273-d632f26be21a None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:27.140754 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9c26580c-76f5-478e-a273-d632f26be21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:27.207240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 497/991] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:27.476013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3add3b4-931c-4275-8992-81095eef5aa9 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:27.580911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3add3b4-931c-4275-8992-81095eef5aa9 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:27.591601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:27.617790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714b601-3886-4237-afab-12ab94842799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:27.619308 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714b601-3886-4237-afab-12ab94842799 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:27.619801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714b601-3886-4237-afab-12ab94842799 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:27.620270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714b601-3886-4237-afab-12ab94842799 None admin] PATH_INFO: `/v3/services/image` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:27.621046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714b601-3886-4237-afab-12ab94842799 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:27.622641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714b601-3886-4237-afab-12ab94842799 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-f714b601-3886-4237-afab-12ab94842799 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:27.631647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:27.632669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:40:27.633531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:27.637811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 495/992] 10.222.0.55 () {56 vars in 1105 bytes} [Thu Sep 8 13:40:27 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:27.639721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26580c-76f5-478e-a273-d632f26be21a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:27.725408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:27.750882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c26580c-76f5-478e-a273-d632f26be21a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:27.785359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c2088-0366-4f87-a262-f598568b11e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:27.785997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c2088-0366-4f87-a262-f598568b11e3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:27.786277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c2088-0366-4f87-a262-f598568b11e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:27.786545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c2088-0366-4f87-a262-f598568b11e3 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:27.787167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c2088-0366-4f87-a262-f598568b11e3 None admin] RBAC: Adding query filter params (name=image) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:27.787467 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c2088-0366-4f87-a262-f598568b11e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:27.789008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c2088-0366-4f87-a262-f598568b11e3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:27.799322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 498/993] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:27.800901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714b601-3886-4237-afab-12ab94842799 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:27.803291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852c233-e4bb-499f-9de4-0429fa2ff11b None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:27.803579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852c233-e4bb-499f-9de4-0429fa2ff11b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:27.803857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852c233-e4bb-499f-9de4-0429fa2ff11b None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:27.805282 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 499/994] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:27.810712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095a579-ac21-4fad-8a1e-1e249d02e0cb None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:27.810996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095a579-ac21-4fad-8a1e-1e249d02e0cb None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:27.811270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095a579-ac21-4fad-8a1e-1e249d02e0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:27.829121 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2095a579-ac21-4fad-8a1e-1e249d02e0cb None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:27.830324 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2095a579-ac21-4fad-8a1e-1e249d02e0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:27.896657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 500/995] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:27.898111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a00677-c43a-452e-8423-61a16e20c629 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:27.898412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a00677-c43a-452e-8423-61a16e20c629 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:27.898759 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a00677-c43a-452e-8423-61a16e20c629 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:27.900900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 501/996] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:27.905785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dfecb-edd2-4f2e-b236-3c27b98434e2 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:27.906160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dfecb-edd2-4f2e-b236-3c27b98434e2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:27.906505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dfecb-edd2-4f2e-b236-3c27b98434e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:27.907881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714b601-3886-4237-afab-12ab94842799 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:27.925423 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d59dfecb-edd2-4f2e-b236-3c27b98434e2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:27.926862 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d59dfecb-edd2-4f2e-b236-3c27b98434e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:27.945392 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00226787-d602-4da8-8778-bb645f4054d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:27.946022 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00226787-d602-4da8-8778-bb645f4054d6 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:27.946294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00226787-d602-4da8-8778-bb645f4054d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:27.946618 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00226787-d602-4da8-8778-bb645f4054d6 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:27.947249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00226787-d602-4da8-8778-bb645f4054d6 None admin] RBAC: Adding query filter params (type=image) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:27.947578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00226787-d602-4da8-8778-bb645f4054d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:27.948899 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00226787-d602-4da8-8778-bb645f4054d6 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:27.954418 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 496/997] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:27 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:28.016564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 502/998] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:28.165534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00226787-d602-4da8-8778-bb645f4054d6 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:28.207197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:28.267265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00226787-d602-4da8-8778-bb645f4054d6 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:28.283676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dfecb-edd2-4f2e-b236-3c27b98434e2 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:28.285865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:28.303230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a323a4-dde1-4b18-94c0-3bdb761af5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:28.303865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a323a4-dde1-4b18-94c0-3bdb761af5a8 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:28.304140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a323a4-dde1-4b18-94c0-3bdb761af5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:28.304405 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a323a4-dde1-4b18-94c0-3bdb761af5a8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:28.304974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a323a4-dde1-4b18-94c0-3bdb761af5a8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:28.306414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a323a4-dde1-4b18-94c0-3bdb761af5a8 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-e1a323a4-dde1-4b18-94c0-3bdb761af5a8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:28.315242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:28.316313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:40:28.317170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:28.322233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 497/999] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:28.323111 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a323a4-dde1-4b18-94c0-3bdb761af5a8 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:28.393615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dfecb-edd2-4f2e-b236-3c27b98434e2 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:28.424105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a323a4-dde1-4b18-94c0-3bdb761af5a8 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:28.425886 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:28.433425 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c237ab9-392b-4adc-8fb2-c0ffe35e5236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:28.434093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c237ab9-392b-4adc-8fb2-c0ffe35e5236 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:28.434437 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c237ab9-392b-4adc-8fb2-c0ffe35e5236 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:28.434696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c237ab9-392b-4adc-8fb2-c0ffe35e5236 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:28.435657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c237ab9-392b-4adc-8fb2-c0ffe35e5236 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:28.437335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c237ab9-392b-4adc-8fb2-c0ffe35e5236 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-0c237ab9-392b-4adc-8fb2-c0ffe35e5236 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:28.447870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:28.448884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:28.450104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 503/1000] 10.222.0.55 () {56 vars in 1126 bytes} [Thu Sep 8 13:40:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:28.454729 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c237ab9-392b-4adc-8fb2-c0ffe35e5236 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:28.465106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388483a-a98d-42a0-b05b-714df0b47537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:28.465712 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388483a-a98d-42a0-b05b-714df0b47537 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:28.466014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388483a-a98d-42a0-b05b-714df0b47537 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:28.466281 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388483a-a98d-42a0-b05b-714df0b47537 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:28.466908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388483a-a98d-42a0-b05b-714df0b47537 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:28.467275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388483a-a98d-42a0-b05b-714df0b47537 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:28.468622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388483a-a98d-42a0-b05b-714df0b47537 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:28.473873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 498/1001] 10.222.0.55 () {56 vars in 1112 bytes} [Thu Sep 8 13:40:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:28.479174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388483a-a98d-42a0-b05b-714df0b47537 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:28.524378 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:28.563292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c237ab9-392b-4adc-8fb2-c0ffe35e5236 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:28.587790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388483a-a98d-42a0-b05b-714df0b47537 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:28.600652 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2bb29-02c4-49e0-ad9f-f2017f4c696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:28.601263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2bb29-02c4-49e0-ad9f-f2017f4c696c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:28.601642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2bb29-02c4-49e0-ad9f-f2017f4c696c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:28.601936 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2bb29-02c4-49e0-ad9f-f2017f4c696c None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:28.602564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2bb29-02c4-49e0-ad9f-f2017f4c696c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:28.602941 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2bb29-02c4-49e0-ad9f-f2017f4c696c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:28.604308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2bb29-02c4-49e0-ad9f-f2017f4c696c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:28.617921 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 504/1002] 10.222.0.55 () {56 vars in 1122 bytes} [Thu Sep 8 13:40:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:28.619621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2bb29-02c4-49e0-ad9f-f2017f4c696c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:28.630598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7e718-3f78-49b5-95b9-260677d5e9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:28.631455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7e718-3f78-49b5-95b9-260677d5e9ef None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:28.631874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7e718-3f78-49b5-95b9-260677d5e9ef None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:28.632296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7e718-3f78-49b5-95b9-260677d5e9ef None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:28.632922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7e718-3f78-49b5-95b9-260677d5e9ef None admin] RBAC: Adding query filter params (type=compute) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:28.633252 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7e718-3f78-49b5-95b9-260677d5e9ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:28.635149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7e718-3f78-49b5-95b9-260677d5e9ef None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:28.640203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 499/1003] 10.222.0.55 () {56 vars in 1112 bytes} [Thu Sep 8 13:40:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:28.646124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7e718-3f78-49b5-95b9-260677d5e9ef None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:28.677764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:28.726169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2bb29-02c4-49e0-ad9f-f2017f4c696c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:28.751569 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7e718-3f78-49b5-95b9-260677d5e9ef None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:28.756014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:28.762979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d46340-7ff3-4485-9fe8-9c77a294be8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:28.763590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d46340-7ff3-4485-9fe8-9c77a294be8b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:28.763867 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d46340-7ff3-4485-9fe8-9c77a294be8b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:28.764137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d46340-7ff3-4485-9fe8-9c77a294be8b None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:28.764754 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d46340-7ff3-4485-9fe8-9c77a294be8b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:28.765077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d46340-7ff3-4485-9fe8-9c77a294be8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:28.766443 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d46340-7ff3-4485-9fe8-9c77a294be8b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:28.773766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 505/1004] 10.222.0.55 () {56 vars in 1122 bytes} [Thu Sep 8 13:40:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:28.777310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d46340-7ff3-4485-9fe8-9c77a294be8b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:28.793337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03158ac9-9a5e-4b0d-bcdc-881ceeb6db16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:28.793970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03158ac9-9a5e-4b0d-bcdc-881ceeb6db16 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:28.794247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03158ac9-9a5e-4b0d-bcdc-881ceeb6db16 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:28.794514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03158ac9-9a5e-4b0d-bcdc-881ceeb6db16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:28.795217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03158ac9-9a5e-4b0d-bcdc-881ceeb6db16 None admin] RBAC: Adding query filter params (interface=public, service_id=d7279e46fb1a4352ac7cc4d65a39601d, region_id=RegionOne) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:28.795520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03158ac9-9a5e-4b0d-bcdc-881ceeb6db16 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:28.796880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03158ac9-9a5e-4b0d-bcdc-881ceeb6db16 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:28.802270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 500/1005] 10.222.0.55 () {56 vars in 1251 bytes} [Thu Sep 8 13:40:28 2022] GET /identity/v3/endpoints?service_id=d7279e46fb1a4352ac7cc4d65a39601d&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:28.808090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03158ac9-9a5e-4b0d-bcdc-881ceeb6db16 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:28.893225 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d46340-7ff3-4485-9fe8-9c77a294be8b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:28.917962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:28.926334 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03158ac9-9a5e-4b0d-bcdc-881ceeb6db16 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:28.932532 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8bc33-3b82-42f2-8d13-c0924f1f55b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:28.933143 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8bc33-3b82-42f2-8d13-c0924f1f55b2 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:28.933420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8bc33-3b82-42f2-8d13-c0924f1f55b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:28.933691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8bc33-3b82-42f2-8d13-c0924f1f55b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:28.934410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8bc33-3b82-42f2-8d13-c0924f1f55b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:28.935803 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8bc33-3b82-42f2-8d13-c0924f1f55b2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:28.953480 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 506/1006] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:40:28 2022] POST /identity/v3/endpoints => generated 325 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:40:28.979206 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08da34d-c239-43d9-9719-e8d264de6b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:28.979832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08da34d-c239-43d9-9719-e8d264de6b23 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:28.980112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08da34d-c239-43d9-9719-e8d264de6b23 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:28.980384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08da34d-c239-43d9-9719-e8d264de6b23 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:28.981006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08da34d-c239-43d9-9719-e8d264de6b23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:28.982523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08da34d-c239-43d9-9719-e8d264de6b23 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:28.998085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 501/1007] 10.222.0.55 () {56 vars in 1087 bytes} [Thu Sep 8 13:40:28 2022] GET /identity/v3/services => generated 1727 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:40:30.121098 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1aec6e-cce6-4002-b700-922fa6d30733 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:30.121648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1aec6e-cce6-4002-b700-922fa6d30733 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:30.121648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1aec6e-cce6-4002-b700-922fa6d30733 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:30.123144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 507/1008] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:30.128292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea75d9-9879-4be2-a599-7ed54dd83ed2 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:30.128670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea75d9-9879-4be2-a599-7ed54dd83ed2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:30.128849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea75d9-9879-4be2-a599-7ed54dd83ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:30.147166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-3eea75d9-9879-4be2-a599-7ed54dd83ed2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:30.148319 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-3eea75d9-9879-4be2-a599-7ed54dd83ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:30.217852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 502/1009] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:30 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:30.479832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1aec6e-cce6-4002-b700-922fa6d30733 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:30.536810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:30.585132 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1aec6e-cce6-4002-b700-922fa6d30733 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:30.618646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d740ee-4f49-4bd5-8ffc-1e608430b7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:30.619249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d740ee-4f49-4bd5-8ffc-1e608430b7c8 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:30.619524 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d740ee-4f49-4bd5-8ffc-1e608430b7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:30.619790 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d740ee-4f49-4bd5-8ffc-1e608430b7c8 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:30.620445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d740ee-4f49-4bd5-8ffc-1e608430b7c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:30.621824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d740ee-4f49-4bd5-8ffc-1e608430b7c8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:30.637760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 508/1010] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:40:30 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:31.158791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c63308-1d8a-4267-922c-f4b4fdff4f8d None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:31.159298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c63308-1d8a-4267-922c-f4b4fdff4f8d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:31.159298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c63308-1d8a-4267-922c-f4b4fdff4f8d None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:31.160648 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 503/1011] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:31.165874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea67bf-fe29-4293-a9a3-4908d6431802 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:31.166253 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea67bf-fe29-4293-a9a3-4908d6431802 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:31.166454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea67bf-fe29-4293-a9a3-4908d6431802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:31.177818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa45ce3-be94-4649-8c0c-50d2e1ae3b27 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:31.178101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa45ce3-be94-4649-8c0c-50d2e1ae3b27 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:31.178380 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa45ce3-be94-4649-8c0c-50d2e1ae3b27 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:31.179693 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 504/1012] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:31.185175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-99ea67bf-fe29-4293-a9a3-4908d6431802 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:31.185473 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9c943-c4a4-4f53-92e9-0ca46bf10e0f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:31.185775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9c943-c4a4-4f53-92e9-0ca46bf10e0f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:31.186054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9c943-c4a4-4f53-92e9-0ca46bf10e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:31.186393 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-99ea67bf-fe29-4293-a9a3-4908d6431802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:31.204751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-0cc9c943-c4a4-4f53-92e9-0ca46bf10e0f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:31.205941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0cc9c943-c4a4-4f53-92e9-0ca46bf10e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:31.265328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 509/1013] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:31 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:31.281793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 505/1014] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:31 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:31.535226 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea67bf-fe29-4293-a9a3-4908d6431802 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:31.545374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9c943-c4a4-4f53-92e9-0ca46bf10e0f None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:31.561060 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:31.578314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:31.641754 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea67bf-fe29-4293-a9a3-4908d6431802 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:31.656390 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9c943-c4a4-4f53-92e9-0ca46bf10e0f None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:31.676581 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31d372-7dc2-4a42-a673-2de0108f3c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:31.677208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31d372-7dc2-4a42-a673-2de0108f3c99 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:31.677486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31d372-7dc2-4a42-a673-2de0108f3c99 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:31.677782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31d372-7dc2-4a42-a673-2de0108f3c99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:31.678892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31d372-7dc2-4a42-a673-2de0108f3c99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:31.680415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31d372-7dc2-4a42-a673-2de0108f3c99 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:31.682480 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 510/1015] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:31.688083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31d372-7dc2-4a42-a673-2de0108f3c99 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:31.697427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcbba6-4d2b-401d-9ad6-55f656d14a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:31.698086 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcbba6-4d2b-401d-9ad6-55f656d14a04 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:31.698366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcbba6-4d2b-401d-9ad6-55f656d14a04 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:31.698635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcbba6-4d2b-401d-9ad6-55f656d14a04 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:31.699212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcbba6-4d2b-401d-9ad6-55f656d14a04 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:31.700556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcbba6-4d2b-401d-9ad6-55f656d14a04 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-83bcbba6-4d2b-401d-9ad6-55f656d14a04 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:31.709309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:31.710517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:31.712167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 506/1016] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:31.716116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcbba6-4d2b-401d-9ad6-55f656d14a04 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:31.794133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31d372-7dc2-4a42-a673-2de0108f3c99 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:31.801131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:31.804391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:31.826223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcbba6-4d2b-401d-9ad6-55f656d14a04 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:31.832366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebe931-af60-40d2-9b2f-87ac754cf4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:31.832985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe931-af60-40d2-9b2f-87ac754cf4b2 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:31.833265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe931-af60-40d2-9b2f-87ac754cf4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:31.833541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebe931-af60-40d2-9b2f-87ac754cf4b2 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:31.834219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebe931-af60-40d2-9b2f-87ac754cf4b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:31.835679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebe931-af60-40d2-9b2f-87ac754cf4b2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:31.863204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dbba54-ea28-4615-9658-58614f88f0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:31.863807 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbba54-ea28-4615-9658-58614f88f0f7 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:31.864089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbba54-ea28-4615-9658-58614f88f0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:31.864359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dbba54-ea28-4615-9658-58614f88f0f7 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:31.864992 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dbba54-ea28-4615-9658-58614f88f0f7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:31.865294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dbba54-ea28-4615-9658-58614f88f0f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:31.866678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dbba54-ea28-4615-9658-58614f88f0f7 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:31.872629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 507/1017] 10.222.0.55 () {56 vars in 1112 bytes} [Thu Sep 8 13:40:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:31.872882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 511/1018] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:40:31 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:40:31.878422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dbba54-ea28-4615-9658-58614f88f0f7 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:32.001350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dbba54-ea28-4615-9658-58614f88f0f7 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:32.037638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93919dc-5a40-4ce8-83fd-2db58d473f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:32.038282 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93919dc-5a40-4ce8-83fd-2db58d473f4f None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:32.038564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93919dc-5a40-4ce8-83fd-2db58d473f4f None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:32.038837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93919dc-5a40-4ce8-83fd-2db58d473f4f None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:32.039480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93919dc-5a40-4ce8-83fd-2db58d473f4f None admin] RBAC: Adding query filter params (type=compute) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:32.039790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93919dc-5a40-4ce8-83fd-2db58d473f4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:32.041192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93919dc-5a40-4ce8-83fd-2db58d473f4f None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:32.047075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 508/1019] 10.222.0.55 () {56 vars in 1112 bytes} [Thu Sep 8 13:40:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:32.052869 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebe931-af60-40d2-9b2f-87ac754cf4b2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:32.159221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebe931-af60-40d2-9b2f-87ac754cf4b2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:32.193879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256acdc7-79c4-40e4-b052-f958e44b6bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:32.194535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256acdc7-79c4-40e4-b052-f958e44b6bc2 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:32.194756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256acdc7-79c4-40e4-b052-f958e44b6bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:32.195029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256acdc7-79c4-40e4-b052-f958e44b6bc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:32.195709 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256acdc7-79c4-40e4-b052-f958e44b6bc2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:32.197101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256acdc7-79c4-40e4-b052-f958e44b6bc2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:32.217999 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 512/1020] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:40:32 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:32.783789 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea7928-d527-430c-a597-e2f21f65a2ff None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:32.784309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea7928-d527-430c-a597-e2f21f65a2ff None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:32.784389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea7928-d527-430c-a597-e2f21f65a2ff None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:32.785786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 509/1021] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:32.791578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0cca7-d434-41a8-9152-d21b21b81286 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:32.791863 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0cca7-d434-41a8-9152-d21b21b81286 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:32.792134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0cca7-d434-41a8-9152-d21b21b81286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:32.812092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-00e0cca7-d434-41a8-9152-d21b21b81286 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:32.813286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-00e0cca7-d434-41a8-9152-d21b21b81286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:32.824894 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:32.890786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 513/1022] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:33.155916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea7928-d527-430c-a597-e2f21f65a2ff None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:33.166151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:33.262258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea7928-d527-430c-a597-e2f21f65a2ff None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:33.296044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df924995-91fe-4d34-a914-74b31c6a5195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:33.296666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df924995-91fe-4d34-a914-74b31c6a5195 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:33.296944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df924995-91fe-4d34-a914-74b31c6a5195 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:33.297216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df924995-91fe-4d34-a914-74b31c6a5195 None admin] PATH_INFO: `/v3/services/image` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:33.297821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df924995-91fe-4d34-a914-74b31c6a5195 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:33.299171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df924995-91fe-4d34-a914-74b31c6a5195 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-df924995-91fe-4d34-a914-74b31c6a5195 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:33.307816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:33.308793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:33.309683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 510/1023] 10.222.0.55 () {56 vars in 1105 bytes} [Thu Sep 8 13:40:33 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:33.314814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0cca7-d434-41a8-9152-d21b21b81286 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:33.410537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0cca7-d434-41a8-9152-d21b21b81286 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:33.444257 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ed3d7-fcf5-48b4-b006-d7cc5105fbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:33.444859 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ed3d7-fcf5-48b4-b006-d7cc5105fbc4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:33.445124 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ed3d7-fcf5-48b4-b006-d7cc5105fbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:33.445392 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ed3d7-fcf5-48b4-b006-d7cc5105fbc4 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:33.446037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ed3d7-fcf5-48b4-b006-d7cc5105fbc4 None admin] RBAC: Adding query filter params (name=image) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:33.446363 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ed3d7-fcf5-48b4-b006-d7cc5105fbc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:33.447714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ed3d7-fcf5-48b4-b006-d7cc5105fbc4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:33.452578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 514/1024] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:33 2022] GET /identity/v3/services?name=image => generated 122 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:33.458069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df924995-91fe-4d34-a914-74b31c6a5195 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:33.530497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:33.562305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df924995-91fe-4d34-a914-74b31c6a5195 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:33.596558 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7be54-8498-4b24-95b7-b9e46e61c746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:33.597156 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7be54-8498-4b24-95b7-b9e46e61c746 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:33.597431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7be54-8498-4b24-95b7-b9e46e61c746 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:33.597697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7be54-8498-4b24-95b7-b9e46e61c746 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:33.598345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7be54-8498-4b24-95b7-b9e46e61c746 None admin] RBAC: Adding query filter params (type=image) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:33.598642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7be54-8498-4b24-95b7-b9e46e61c746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:33.599985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7be54-8498-4b24-95b7-b9e46e61c746 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:33.604958 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 511/1025] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:33 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:33.610635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ed3d7-fcf5-48b4-b006-d7cc5105fbc4 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:33.619356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:33.711197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ed3d7-fcf5-48b4-b006-d7cc5105fbc4 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:33.744231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b91fe69-afc3-4820-8a13-acbc61eefcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:33.744833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91fe69-afc3-4820-8a13-acbc61eefcbb None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:33.745104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91fe69-afc3-4820-8a13-acbc61eefcbb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:33.745368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b91fe69-afc3-4820-8a13-acbc61eefcbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:33.746084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91fe69-afc3-4820-8a13-acbc61eefcbb None admin] RBAC: Adding query filter params (interface=public, service_id=ff58628bc9df441b876ece4b4f8e8caa, region_id=RegionOne) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:33.746386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91fe69-afc3-4820-8a13-acbc61eefcbb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:33.747732 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b91fe69-afc3-4820-8a13-acbc61eefcbb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:33.752808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 515/1026] 10.222.0.55 () {56 vars in 1251 bytes} [Thu Sep 8 13:40:33 2022] GET /identity/v3/endpoints?service_id=ff58628bc9df441b876ece4b4f8e8caa&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:33.758370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7be54-8498-4b24-95b7-b9e46e61c746 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:33.856420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7be54-8498-4b24-95b7-b9e46e61c746 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:33.889103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80a02a5-048b-4ba4-b5d9-0301d7cdabc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:33.889700 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a02a5-048b-4ba4-b5d9-0301d7cdabc5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:33.890002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a02a5-048b-4ba4-b5d9-0301d7cdabc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:33.890269 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80a02a5-048b-4ba4-b5d9-0301d7cdabc5 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:33.890867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a02a5-048b-4ba4-b5d9-0301d7cdabc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:33.892202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80a02a5-048b-4ba4-b5d9-0301d7cdabc5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:33.893209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:33.907792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 512/1027] 10.222.0.55 () {56 vars in 1087 bytes} [Thu Sep 8 13:40:33 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:40:34.130960 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d46196-2953-4fcd-a2b7-de4b9f5c4643 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:34.131353 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d46196-2953-4fcd-a2b7-de4b9f5c4643 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:34.131534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d46196-2953-4fcd-a2b7-de4b9f5c4643 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:34.132981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 516/1028] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:34.138263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c496b5-d992-4915-841a-49f15f2ba1bc None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:34.138552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c496b5-d992-4915-841a-49f15f2ba1bc None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:34.138826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c496b5-d992-4915-841a-49f15f2ba1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:34.158027 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-09c496b5-d992-4915-841a-49f15f2ba1bc None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:34.159203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-09c496b5-d992-4915-841a-49f15f2ba1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:34.227388 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 513/1029] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:34.475982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bffa6-1bfb-4586-bab1-e297026a5238 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:34.476469 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bffa6-1bfb-4586-bab1-e297026a5238 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:34.476469 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bffa6-1bfb-4586-bab1-e297026a5238 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:34.477945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 517/1030] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:34.482958 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30685c-4080-4062-8499-ce98cb6e3c51 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:34.483244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30685c-4080-4062-8499-ce98cb6e3c51 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:34.483514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30685c-4080-4062-8499-ce98cb6e3c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:34.500150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bffa6-1bfb-4586-bab1-e297026a5238 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:34.501848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-5f30685c-4080-4062-8499-ce98cb6e3c51 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:34.503020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5f30685c-4080-4062-8499-ce98cb6e3c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:34.573095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:34.574235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 514/1031] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:34.607402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bffa6-1bfb-4586-bab1-e297026a5238 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:34.641304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1233a4-f751-4fb0-82b9-aed5d9d78108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:34.641962 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1233a4-f751-4fb0-82b9-aed5d9d78108 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:34.642241 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1233a4-f751-4fb0-82b9-aed5d9d78108 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:34.642510 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1233a4-f751-4fb0-82b9-aed5d9d78108 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:34.643629 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1233a4-f751-4fb0-82b9-aed5d9d78108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:34.645054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1233a4-f751-4fb0-82b9-aed5d9d78108 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:34.646830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 518/1032] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:34.652473 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30685c-4080-4062-8499-ce98cb6e3c51 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:34.713299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:34.759355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30685c-4080-4062-8499-ce98cb6e3c51 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:34.798103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122febee-aa5a-4ce3-99f4-08221129000e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:34.798714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122febee-aa5a-4ce3-99f4-08221129000e None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:34.798997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122febee-aa5a-4ce3-99f4-08221129000e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:34.799270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122febee-aa5a-4ce3-99f4-08221129000e None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:34.799926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122febee-aa5a-4ce3-99f4-08221129000e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:34.801354 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122febee-aa5a-4ce3-99f4-08221129000e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:34.829755 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 515/1033] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:40:34 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:34.852201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1233a4-f751-4fb0-82b9-aed5d9d78108 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:34.976064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1233a4-f751-4fb0-82b9-aed5d9d78108 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:35.013147 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379bb26-a6a9-4899-9ee5-dadbb93cf863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:35.013702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379bb26-a6a9-4899-9ee5-dadbb93cf863 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:35.014012 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379bb26-a6a9-4899-9ee5-dadbb93cf863 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:35.014287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379bb26-a6a9-4899-9ee5-dadbb93cf863 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:35.021409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:35.026680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379bb26-a6a9-4899-9ee5-dadbb93cf863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-2379bb26-a6a9-4899-9ee5-dadbb93cf863 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:35.031874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:35.033015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:35.034029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:35.034029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:35.034029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:35.034029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:35.034029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:35.034029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:35.034029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:35.034029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:35.034029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:35.034029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:35.034029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:35.034029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:35.034029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 519/1034] 10.222.0.55 () {56 vars in 1120 bytes} [Thu Sep 8 13:40:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:35.039170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122febee-aa5a-4ce3-99f4-08221129000e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:35.153676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122febee-aa5a-4ce3-99f4-08221129000e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:35.168664 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:35.190988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b24312-bfbc-4766-a8f6-c548282e212e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:35.191604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b24312-bfbc-4766-a8f6-c548282e212e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:35.191886 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b24312-bfbc-4766-a8f6-c548282e212e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:35.192159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b24312-bfbc-4766-a8f6-c548282e212e None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:35.192827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b24312-bfbc-4766-a8f6-c548282e212e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:35.193421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b24312-bfbc-4766-a8f6-c548282e212e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:35.194808 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b24312-bfbc-4766-a8f6-c548282e212e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:35.204032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 516/1035] 10.222.0.55 () {56 vars in 1115 bytes} [Thu Sep 8 13:40:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:35.210007 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379bb26-a6a9-4899-9ee5-dadbb93cf863 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:35.310584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379bb26-a6a9-4899-9ee5-dadbb93cf863 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:35.342947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32697c-d232-4f0c-8544-5c46df6bcc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:35.343570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32697c-d232-4f0c-8544-5c46df6bcc8b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:35.343843 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32697c-d232-4f0c-8544-5c46df6bcc8b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:35.344108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32697c-d232-4f0c-8544-5c46df6bcc8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:35.354963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32697c-d232-4f0c-8544-5c46df6bcc8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:35.356377 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32697c-d232-4f0c-8544-5c46df6bcc8b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-de32697c-d232-4f0c-8544-5c46df6bcc8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:35.370174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:35.371109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:35.372056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:35.372056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:35.372056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:35.373799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 520/1036] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:35.377002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b24312-bfbc-4766-a8f6-c548282e212e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:35.483619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b24312-bfbc-4766-a8f6-c548282e212e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:35.516608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d2b12-aaa2-4c08-b989-c9e134d83111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:35.517208 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d2b12-aaa2-4c08-b989-c9e134d83111 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:35.517488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d2b12-aaa2-4c08-b989-c9e134d83111 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:35.517785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d2b12-aaa2-4c08-b989-c9e134d83111 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:35.518433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d2b12-aaa2-4c08-b989-c9e134d83111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:35.518733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d2b12-aaa2-4c08-b989-c9e134d83111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:35.520093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d2b12-aaa2-4c08-b989-c9e134d83111 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:35.536359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 517/1037] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:35.537537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32697c-d232-4f0c-8544-5c46df6bcc8b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:35.588691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:35.640111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32697c-d232-4f0c-8544-5c46df6bcc8b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:35.674203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44504783-aa62-4ba2-8df1-e212470500e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:35.674834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504783-aa62-4ba2-8df1-e212470500e8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:35.675114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504783-aa62-4ba2-8df1-e212470500e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:35.675380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504783-aa62-4ba2-8df1-e212470500e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:35.682914 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44504783-aa62-4ba2-8df1-e212470500e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:35.684348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44504783-aa62-4ba2-8df1-e212470500e8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-44504783-aa62-4ba2-8df1-e212470500e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:40:35.695817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:35.696742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:35.697817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:35.697817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:35.697817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:40:35.697817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:40:35.697817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:40:35.697817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:40:35.697817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:40:35.697817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:40:35.697817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:40:35.697817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:35.697817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 521/1038] 10.222.0.55 () {56 vars in 1093 bytes} [Thu Sep 8 13:40:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:35.704219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d2b12-aaa2-4c08-b989-c9e134d83111 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:35.740399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:35.808578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d2b12-aaa2-4c08-b989-c9e134d83111 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:35.842632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2c470-e4c1-406d-b457-27b3e88224b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:35.843164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2c470-e4c1-406d-b457-27b3e88224b7 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:35.843444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2c470-e4c1-406d-b457-27b3e88224b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:35.843714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2c470-e4c1-406d-b457-27b3e88224b7 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:35.844437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2c470-e4c1-406d-b457-27b3e88224b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:35.844741 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2c470-e4c1-406d-b457-27b3e88224b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:35.846183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2c470-e4c1-406d-b457-27b3e88224b7 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:35.867577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 518/1039] 10.222.0.55 () {56 vars in 1133 bytes} [Thu Sep 8 13:40:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:35.868917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44504783-aa62-4ba2-8df1-e212470500e8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:35.982455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44504783-aa62-4ba2-8df1-e212470500e8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:35.993590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:36.018931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52eeeb7-e256-416a-8d48-a1f9fbaedc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:36.019534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eeeb7-e256-416a-8d48-a1f9fbaedc67 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.019805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eeeb7-e256-416a-8d48-a1f9fbaedc67 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.020077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eeeb7-e256-416a-8d48-a1f9fbaedc67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.078293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e79f05-184a-4e37-8320-f133edd73018 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.078837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e79f05-184a-4e37-8320-f133edd73018 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.078837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e79f05-184a-4e37-8320-f133edd73018 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.080125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 519/1040] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:36.085125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359f279-e715-44eb-889c-d88f9b509ddb None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.085412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359f279-e715-44eb-889c-d88f9b509ddb None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.085689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359f279-e715-44eb-889c-d88f9b509ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.087414 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52eeeb7-e256-416a-8d48-a1f9fbaedc67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:36.088927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52eeeb7-e256-416a-8d48-a1f9fbaedc67 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:36.105606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-8359f279-e715-44eb-889c-d88f9b509ddb None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:36.107256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8359f279-e715-44eb-889c-d88f9b509ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:36.183414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 520/1041] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:36.197836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 522/1042] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:35 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:36.203962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359f279-e715-44eb-889c-d88f9b509ddb None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:36.229904 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:36.304786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359f279-e715-44eb-889c-d88f9b509ddb None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:36.338774 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12552cf2-94af-4ba8-87dc-c5747ea6c3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:36.339395 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12552cf2-94af-4ba8-87dc-c5747ea6c3b3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.339672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12552cf2-94af-4ba8-87dc-c5747ea6c3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.339941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12552cf2-94af-4ba8-87dc-c5747ea6c3b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.349842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12552cf2-94af-4ba8-87dc-c5747ea6c3b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:36.351218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12552cf2-94af-4ba8-87dc-c5747ea6c3b3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-12552cf2-94af-4ba8-87dc-c5747ea6c3b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:36.365449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:36.366469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:36.367425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:36.367425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:36.367425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:36.370061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 521/1043] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:36.372653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52eeeb7-e256-416a-8d48-a1f9fbaedc67 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:36.384170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:36.449592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12552cf2-94af-4ba8-87dc-c5747ea6c3b3 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:36.473249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52eeeb7-e256-416a-8d48-a1f9fbaedc67 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:36.514117 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc5954-3633-4e4e-807d-39927988ed1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:36.514747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc5954-3633-4e4e-807d-39927988ed1a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.515010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc5954-3633-4e4e-807d-39927988ed1a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.515284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc5954-3633-4e4e-807d-39927988ed1a None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.515934 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc5954-3633-4e4e-807d-39927988ed1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:36.516226 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc5954-3633-4e4e-807d-39927988ed1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:36.517594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc5954-3633-4e4e-807d-39927988ed1a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:36.529463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 523/1044] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:36.533280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:36.535146 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc5954-3633-4e4e-807d-39927988ed1a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:36.559221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12552cf2-94af-4ba8-87dc-c5747ea6c3b3 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:36.597292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4662235-6f7c-4875-a98f-9aac2de951a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:36.597939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4662235-6f7c-4875-a98f-9aac2de951a8 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.598216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4662235-6f7c-4875-a98f-9aac2de951a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.598488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4662235-6f7c-4875-a98f-9aac2de951a8 None admin] PATH_INFO: `/v3/services/image` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.599066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4662235-6f7c-4875-a98f-9aac2de951a8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:36.600412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4662235-6f7c-4875-a98f-9aac2de951a8 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:36.604924 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-d4662235-6f7c-4875-a98f-9aac2de951a8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:36.610588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:36.611600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:36.612516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 522/1045] 10.222.0.55 () {56 vars in 1105 bytes} [Thu Sep 8 13:40:36 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:36.618435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4662235-6f7c-4875-a98f-9aac2de951a8 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:36.640768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc5954-3633-4e4e-807d-39927988ed1a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:36.676373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6de61-cadd-42ce-87d7-86258301cb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:36.677017 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6de61-cadd-42ce-87d7-86258301cb29 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.677331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6de61-cadd-42ce-87d7-86258301cb29 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.677558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6de61-cadd-42ce-87d7-86258301cb29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.684700 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6de61-cadd-42ce-87d7-86258301cb29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:36.686293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6de61-cadd-42ce-87d7-86258301cb29 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-cdc6de61-cadd-42ce-87d7-86258301cb29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:40:36.698399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:36.699336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:36.700227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:40:36.700227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:40:36.700227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:36.700227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:36.700227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:36.700227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:36.700227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:36.700227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:36.702047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 524/1046] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:36.705170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6de61-cadd-42ce-87d7-86258301cb29 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:36.726584 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4662235-6f7c-4875-a98f-9aac2de951a8 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:36.762788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eafa67-4b43-44ca-ac9b-76dd44c7b967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:36.763401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eafa67-4b43-44ca-ac9b-76dd44c7b967 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.763684 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eafa67-4b43-44ca-ac9b-76dd44c7b967 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.763956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eafa67-4b43-44ca-ac9b-76dd44c7b967 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.764757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eafa67-4b43-44ca-ac9b-76dd44c7b967 None admin] RBAC: Adding query filter params (name=image) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:36.765065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eafa67-4b43-44ca-ac9b-76dd44c7b967 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:36.766812 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eafa67-4b43-44ca-ac9b-76dd44c7b967 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:36.773775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 523/1047] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:36.778360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eafa67-4b43-44ca-ac9b-76dd44c7b967 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:36.783991 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:36.819662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6de61-cadd-42ce-87d7-86258301cb29 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:36.850030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:36.856891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da372f5-aff9-44da-aec2-a53d46ed92f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:36.857499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da372f5-aff9-44da-aec2-a53d46ed92f1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.857810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da372f5-aff9-44da-aec2-a53d46ed92f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.858087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da372f5-aff9-44da-aec2-a53d46ed92f1 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.858767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da372f5-aff9-44da-aec2-a53d46ed92f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:36.859067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da372f5-aff9-44da-aec2-a53d46ed92f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:36.860504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da372f5-aff9-44da-aec2-a53d46ed92f1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:36.878606 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 525/1048] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:40:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:36.879584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da372f5-aff9-44da-aec2-a53d46ed92f1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:36.902791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eafa67-4b43-44ca-ac9b-76dd44c7b967 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:36.944754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb31168-6137-42ff-a477-0fb887621625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:36.945451 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb31168-6137-42ff-a477-0fb887621625 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.945825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb31168-6137-42ff-a477-0fb887621625 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.946218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb31168-6137-42ff-a477-0fb887621625 None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.947429 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb31168-6137-42ff-a477-0fb887621625 None admin] RBAC: Adding query filter params (type=image) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:36.947800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb31168-6137-42ff-a477-0fb887621625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:36.951877 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb31168-6137-42ff-a477-0fb887621625 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:36.958596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 524/1049] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:36 2022] GET /identity/v3/services?type=image => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:36.964584 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb31168-6137-42ff-a477-0fb887621625 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:36.988831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da372f5-aff9-44da-aec2-a53d46ed92f1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:37.025085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da16e0-18a7-4fda-9a9a-68da33570c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:37.025689 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da16e0-18a7-4fda-9a9a-68da33570c52 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:37.025998 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da16e0-18a7-4fda-9a9a-68da33570c52 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:37.026273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da16e0-18a7-4fda-9a9a-68da33570c52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:37.026996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da16e0-18a7-4fda-9a9a-68da33570c52 None admin] RBAC: Adding query filter params (role.id=a50724aa8b014431acb0d653bf1e41e7, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=6ffa73f81539472391983c51447af1b6) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:37.027305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da16e0-18a7-4fda-9a9a-68da33570c52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:37.028735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70da16e0-18a7-4fda-9a9a-68da33570c52 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:37.034476 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 526/1050] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:36 2022] GET /identity/v3/role_assignments?user.id=6ffa73f81539472391983c51447af1b6&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=a50724aa8b014431acb0d653bf1e41e7 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:37.039040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:37.078989 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb31168-6137-42ff-a477-0fb887621625 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:37.085663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87ee42-2304-43b2-8961-c57a5609f0b5 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:37.085995 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87ee42-2304-43b2-8961-c57a5609f0b5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:37.086257 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87ee42-2304-43b2-8961-c57a5609f0b5 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:37.088767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 527/1051] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:37.094653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecefa2e-65a5-48f1-8da2-109df0004e79 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:37.094930 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecefa2e-65a5-48f1-8da2-109df0004e79 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:37.095198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecefa2e-65a5-48f1-8da2-109df0004e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:37.114287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1ecefa2e-65a5-48f1-8da2-109df0004e79 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:37.115507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1ecefa2e-65a5-48f1-8da2-109df0004e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:37.119027 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b068975-f9a4-4274-97ce-b3add9f490f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:37.119868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b068975-f9a4-4274-97ce-b3add9f490f8 None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:37.120151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b068975-f9a4-4274-97ce-b3add9f490f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:37.120420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b068975-f9a4-4274-97ce-b3add9f490f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:37.121108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b068975-f9a4-4274-97ce-b3add9f490f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:37.122517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b068975-f9a4-4274-97ce-b3add9f490f8 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:37.150721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 525/1052] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:40:36 2022] POST /identity/v3/endpoints => generated 327 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:40:37.182194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:37.212268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 528/1053] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:37.486157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b068975-f9a4-4274-97ce-b3add9f490f8 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:37.589600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b068975-f9a4-4274-97ce-b3add9f490f8 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:37.612729 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:37.625701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb577f-0627-45a3-8a5f-c598d28deb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:37.626361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb577f-0627-45a3-8a5f-c598d28deb8c None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:37.626634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb577f-0627-45a3-8a5f-c598d28deb8c None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:37.626899 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb577f-0627-45a3-8a5f-c598d28deb8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:37.637448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb577f-0627-45a3-8a5f-c598d28deb8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-90fb577f-0627-45a3-8a5f-c598d28deb8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:37.642186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:37.643130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:37.643981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:37.643981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:37.643981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:37.643981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:37.643981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:37.643981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:37.643981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:37.643981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:37.643981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:37.643981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:37.643981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:37.643981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:37.643981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 526/1054] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:37.651309 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecefa2e-65a5-48f1-8da2-109df0004e79 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:37.750838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecefa2e-65a5-48f1-8da2-109df0004e79 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:37.784610 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99669e8-7981-4687-800e-37e86d380db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:37.785215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99669e8-7981-4687-800e-37e86d380db5 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:37.785491 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99669e8-7981-4687-800e-37e86d380db5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:37.785783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99669e8-7981-4687-800e-37e86d380db5 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:37.786444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99669e8-7981-4687-800e-37e86d380db5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:37.786743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99669e8-7981-4687-800e-37e86d380db5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:37.788133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99669e8-7981-4687-800e-37e86d380db5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:37.801694 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 529/1055] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:37.802940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb577f-0627-45a3-8a5f-c598d28deb8c None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:37.911790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb577f-0627-45a3-8a5f-c598d28deb8c None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:37.949431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e483898-3fd0-43ab-9a9d-05be944db084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:37.950085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e483898-3fd0-43ab-9a9d-05be944db084 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:37.950366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e483898-3fd0-43ab-9a9d-05be944db084 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:37.950638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e483898-3fd0-43ab-9a9d-05be944db084 None admin] PATH_INFO: `/v3/users/313377cf08c54e959473499b18db7acc` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:37.964370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e483898-3fd0-43ab-9a9d-05be944db084 None admin] RBAC: Authorizing `identity:get_user(user_id=313377cf08c54e959473499b18db7acc)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:37.965826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e483898-3fd0-43ab-9a9d-05be944db084 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:37.967594 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 527/1056] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:37 2022] GET /identity/v3/users/313377cf08c54e959473499b18db7acc => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:37.973165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99669e8-7981-4687-800e-37e86d380db5 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:38.018084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:38.078731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99669e8-7981-4687-800e-37e86d380db5 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:38.111540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cb624-c2f0-432e-ac8c-7323d5b86e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:38.112139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cb624-c2f0-432e-ac8c-7323d5b86e1f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:38.112415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cb624-c2f0-432e-ac8c-7323d5b86e1f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:38.112689 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cb624-c2f0-432e-ac8c-7323d5b86e1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:38.179879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cb624-c2f0-432e-ac8c-7323d5b86e1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:38.181899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cb624-c2f0-432e-ac8c-7323d5b86e1f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:38.221880 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:38.282806 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 530/1057] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:38.288888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e483898-3fd0-43ab-9a9d-05be944db084 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:38.334719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:38.393102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e483898-3fd0-43ab-9a9d-05be944db084 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:38.428260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae687da-3087-4e54-8482-3926b09594b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:38.428891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae687da-3087-4e54-8482-3926b09594b3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:38.429169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae687da-3087-4e54-8482-3926b09594b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:38.429442 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae687da-3087-4e54-8482-3926b09594b3 None admin] PATH_INFO: `/v3/projects/ae8123a373ac45a3914597a54687e842` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:38.434137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae687da-3087-4e54-8482-3926b09594b3 None admin] RBAC: Authorizing `identity:get_project(project_id=ae8123a373ac45a3914597a54687e842)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:38.434137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae687da-3087-4e54-8482-3926b09594b3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:38.434137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 528/1058] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:38 2022] GET /identity/v3/projects/ae8123a373ac45a3914597a54687e842 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:38.440219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cb624-c2f0-432e-ac8c-7323d5b86e1f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:38.541867 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cb624-c2f0-432e-ac8c-7323d5b86e1f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:38.575099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061990e8-b0f6-46ba-9c60-9d638e474a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:38.575695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061990e8-b0f6-46ba-9c60-9d638e474a7e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:38.575975 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061990e8-b0f6-46ba-9c60-9d638e474a7e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:38.576245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061990e8-b0f6-46ba-9c60-9d638e474a7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:38.576987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061990e8-b0f6-46ba-9c60-9d638e474a7e None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=ae8123a373ac45a3914597a54687e842, user.id=313377cf08c54e959473499b18db7acc) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:38.577295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061990e8-b0f6-46ba-9c60-9d638e474a7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:38.578730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061990e8-b0f6-46ba-9c60-9d638e474a7e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:38.589980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 531/1059] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:38 2022] GET /identity/v3/role_assignments?user.id=313377cf08c54e959473499b18db7acc&scope.project.id=ae8123a373ac45a3914597a54687e842&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:39.212676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f7cbc-0e5f-47b2-b146-66b5451ad5d3 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:39.212870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f7cbc-0e5f-47b2-b146-66b5451ad5d3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:39.213137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f7cbc-0e5f-47b2-b146-66b5451ad5d3 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:39.214779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 529/1060] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:39.220338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064f0c5-3787-4790-88b8-15f376205721 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:39.220857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064f0c5-3787-4790-88b8-15f376205721 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:39.220933 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064f0c5-3787-4790-88b8-15f376205721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:39.234610 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:39.240737 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2064f0c5-3787-4790-88b8-15f376205721 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:39.241956 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2064f0c5-3787-4790-88b8-15f376205721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:39.320820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 532/1061] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:39.333758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e6118-9357-495d-93dc-019679d5a996 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:39.333758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e6118-9357-495d-93dc-019679d5a996 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:39.333758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e6118-9357-495d-93dc-019679d5a996 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:39.333758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 530/1062] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:39.336339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce66ca9-8a8d-46f5-b02e-b2ab13908c33 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:39.337137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce66ca9-8a8d-46f5-b02e-b2ab13908c33 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:39.337411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce66ca9-8a8d-46f5-b02e-b2ab13908c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:39.356053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4ce66ca9-8a8d-46f5-b02e-b2ab13908c33 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:39.357229 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4ce66ca9-8a8d-46f5-b02e-b2ab13908c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:39.424368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 533/1063] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:39.597011 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e6118-9357-495d-93dc-019679d5a996 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:39.697742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e6118-9357-495d-93dc-019679d5a996 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:39.703318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce66ca9-8a8d-46f5-b02e-b2ab13908c33 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:39.708904 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:39.734922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136fb6f-9a3f-4242-a6fe-a5714e496325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:39.735556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136fb6f-9a3f-4242-a6fe-a5714e496325 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:39.735813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136fb6f-9a3f-4242-a6fe-a5714e496325 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:39.736081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136fb6f-9a3f-4242-a6fe-a5714e496325 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:39.747062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136fb6f-9a3f-4242-a6fe-a5714e496325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-3136fb6f-9a3f-4242-a6fe-a5714e496325 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:39.751794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:39.752757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:39.753644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:39.753644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:39.753644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:39.753644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:39.753644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:39.753644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:39.753644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:39.753644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:39.753644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:39.753644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:39.753644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:39.753644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:39.753644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 531/1064] 10.222.0.55 () {56 vars in 1120 bytes} [Thu Sep 8 13:40:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:39.758906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136fb6f-9a3f-4242-a6fe-a5714e496325 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:39.772777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:39.809232 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce66ca9-8a8d-46f5-b02e-b2ab13908c33 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:39.844248 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275bdc4-ed33-4e71-8fa1-877d16712b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:39.844857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275bdc4-ed33-4e71-8fa1-877d16712b8c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:39.845138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275bdc4-ed33-4e71-8fa1-877d16712b8c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:39.845409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275bdc4-ed33-4e71-8fa1-877d16712b8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:39.871311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136fb6f-9a3f-4242-a6fe-a5714e496325 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:39.916226 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275bdc4-ed33-4e71-8fa1-877d16712b8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:39.916363 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e36f91-3cd8-4364-9020-1f0fea168e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:39.916943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e36f91-3cd8-4364-9020-1f0fea168e25 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:39.917216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e36f91-3cd8-4364-9020-1f0fea168e25 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:39.917480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e36f91-3cd8-4364-9020-1f0fea168e25 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:39.917676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275bdc4-ed33-4e71-8fa1-877d16712b8c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:39.918175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e36f91-3cd8-4364-9020-1f0fea168e25 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:39.918475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e36f91-3cd8-4364-9020-1f0fea168e25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:39.919988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e36f91-3cd8-4364-9020-1f0fea168e25 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:39.930893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 532/1065] 10.222.0.55 () {56 vars in 1115 bytes} [Thu Sep 8 13:40:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:39.936381 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e36f91-3cd8-4364-9020-1f0fea168e25 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:39.969753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:39.983918 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:40.021467 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 534/1066] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:40:40.027731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275bdc4-ed33-4e71-8fa1-877d16712b8c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:40.047849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e36f91-3cd8-4364-9020-1f0fea168e25 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:40.084519 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525d719-9a2e-46d4-b311-dad5ab9a9671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:40.085316 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525d719-9a2e-46d4-b311-dad5ab9a9671 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:40.085430 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525d719-9a2e-46d4-b311-dad5ab9a9671 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:40.085658 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0525d719-9a2e-46d4-b311-dad5ab9a9671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:40.096949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525d719-9a2e-46d4-b311-dad5ab9a9671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:40.098414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0525d719-9a2e-46d4-b311-dad5ab9a9671 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-0525d719-9a2e-46d4-b311-dad5ab9a9671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:40.113317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:40.114341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:40.115247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:40.115247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:40.115247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:40.118197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 533/1067] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:40.121104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525d719-9a2e-46d4-b311-dad5ab9a9671 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:40.131737 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275bdc4-ed33-4e71-8fa1-877d16712b8c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:40.167593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894d8a4-6ccf-4ac3-9157-0ed8a54c83e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:40.168289 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894d8a4-6ccf-4ac3-9157-0ed8a54c83e1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:40.168543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894d8a4-6ccf-4ac3-9157-0ed8a54c83e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:40.168809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894d8a4-6ccf-4ac3-9157-0ed8a54c83e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:40.178694 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894d8a4-6ccf-4ac3-9157-0ed8a54c83e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:40.180155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894d8a4-6ccf-4ac3-9157-0ed8a54c83e1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-5894d8a4-6ccf-4ac3-9157-0ed8a54c83e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:40.195233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:40.197044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:40.198202 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:40.198202 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:40.198202 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:40.201900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 535/1068] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:40.202835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894d8a4-6ccf-4ac3-9157-0ed8a54c83e1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:40.215134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:40.223582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:40.233018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0525d719-9a2e-46d4-b311-dad5ab9a9671 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:40.272000 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7209f-15c0-4bdc-a49a-2ed4e61ff589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:40.272855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7209f-15c0-4bdc-a49a-2ed4e61ff589 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:40.273145 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7209f-15c0-4bdc-a49a-2ed4e61ff589 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:40.273416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7209f-15c0-4bdc-a49a-2ed4e61ff589 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:40.274770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7209f-15c0-4bdc-a49a-2ed4e61ff589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:40.275163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7209f-15c0-4bdc-a49a-2ed4e61ff589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:40.276750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a7209f-15c0-4bdc-a49a-2ed4e61ff589 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:40.289487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 534/1069] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:40.295556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7209f-15c0-4bdc-a49a-2ed4e61ff589 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:40.314853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894d8a4-6ccf-4ac3-9157-0ed8a54c83e1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:40.349664 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd91abc-d8a3-4114-bdf6-b45496a6b753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:40.350753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd91abc-d8a3-4114-bdf6-b45496a6b753 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:40.351218 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd91abc-d8a3-4114-bdf6-b45496a6b753 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:40.351689 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd91abc-d8a3-4114-bdf6-b45496a6b753 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:40.353050 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd91abc-d8a3-4114-bdf6-b45496a6b753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:40.353689 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd91abc-d8a3-4114-bdf6-b45496a6b753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:40.355336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd91abc-d8a3-4114-bdf6-b45496a6b753 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:40.369139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 536/1070] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:40.422897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a7209f-15c0-4bdc-a49a-2ed4e61ff589 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:40.463759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d560c-5cc8-41b4-9be4-abdf7f127ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:40.464784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d560c-5cc8-41b4-9be4-abdf7f127ec3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:40.465242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d560c-5cc8-41b4-9be4-abdf7f127ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:40.465687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d560c-5cc8-41b4-9be4-abdf7f127ec3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:40.474112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d560c-5cc8-41b4-9be4-abdf7f127ec3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:40.475897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d560c-5cc8-41b4-9be4-abdf7f127ec3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-208d560c-5cc8-41b4-9be4-abdf7f127ec3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:40:40.489415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:40.490524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:40.498632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:40.498632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:40.498632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:40:40.498632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:40:40.498632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:40:40.498632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:40:40.498632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:40:40.498632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:40:40.498632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:40:40.498632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:40.498632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 535/1071] 10.222.0.55 () {56 vars in 1093 bytes} [Thu Sep 8 13:40:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:40.502389 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd91abc-d8a3-4114-bdf6-b45496a6b753 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:40.588250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:40.613280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd91abc-d8a3-4114-bdf6-b45496a6b753 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:40.651858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab30690-9f8a-40b6-9c4e-dcd28295e6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:40.652484 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab30690-9f8a-40b6-9c4e-dcd28295e6ed None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:40.652763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab30690-9f8a-40b6-9c4e-dcd28295e6ed None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:40.653033 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab30690-9f8a-40b6-9c4e-dcd28295e6ed None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:40.653850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab30690-9f8a-40b6-9c4e-dcd28295e6ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:40.654637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab30690-9f8a-40b6-9c4e-dcd28295e6ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:40.655573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab30690-9f8a-40b6-9c4e-dcd28295e6ed None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:40.674890 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 537/1072] 10.222.0.55 () {56 vars in 1133 bytes} [Thu Sep 8 13:40:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:40.681235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d560c-5cc8-41b4-9be4-abdf7f127ec3 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:40.686015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:40.723962 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4939a-5ee1-446f-9639-a382ac08e495 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:40.724243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4939a-5ee1-446f-9639-a382ac08e495 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:40.724519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4939a-5ee1-446f-9639-a382ac08e495 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:40.725894 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 538/1073] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:40.730796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c810b-f6e2-4eeb-aa83-a003eff5da8d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:40.731075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c810b-f6e2-4eeb-aa83-a003eff5da8d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:40.731422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c810b-f6e2-4eeb-aa83-a003eff5da8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:40.749317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6d6c810b-f6e2-4eeb-aa83-a003eff5da8d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:40.750517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6d6c810b-f6e2-4eeb-aa83-a003eff5da8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:40.786825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d560c-5cc8-41b4-9be4-abdf7f127ec3 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:40.817127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 539/1074] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:40.826859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c5c2c-656a-4d7a-9c69-f533e01c2623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:40.827490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c5c2c-656a-4d7a-9c69-f533e01c2623 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:40.827769 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c5c2c-656a-4d7a-9c69-f533e01c2623 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:40.828040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c5c2c-656a-4d7a-9c69-f533e01c2623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:40.838435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c5c2c-656a-4d7a-9c69-f533e01c2623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:40.839860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c5c2c-656a-4d7a-9c69-f533e01c2623 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-ed3c5c2c-656a-4d7a-9c69-f533e01c2623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:40.853627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:40.854585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:40.855501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:40.855501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:40.855501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:40.855501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 536/1075] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:40.860611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c810b-f6e2-4eeb-aa83-a003eff5da8d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:40.924069 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:40.959082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c810b-f6e2-4eeb-aa83-a003eff5da8d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:40.993498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6ef2a-0ca1-406b-8cf3-79d2ba5af106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:40.994124 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6ef2a-0ca1-406b-8cf3-79d2ba5af106 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:40.994401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6ef2a-0ca1-406b-8cf3-79d2ba5af106 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:40.994669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea6ef2a-0ca1-406b-8cf3-79d2ba5af106 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:40.995301 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6ef2a-0ca1-406b-8cf3-79d2ba5af106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:40.995604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6ef2a-0ca1-406b-8cf3-79d2ba5af106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:40.996982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea6ef2a-0ca1-406b-8cf3-79d2ba5af106 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:41.007855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 540/1076] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:41.013538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c5c2c-656a-4d7a-9c69-f533e01c2623 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:41.068972 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:41.092358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6ef2a-0ca1-406b-8cf3-79d2ba5af106 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:41.119657 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c5c2c-656a-4d7a-9c69-f533e01c2623 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:41.152375 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef226dc-d7dc-454d-a1be-fd9b22d60181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:41.152990 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef226dc-d7dc-454d-a1be-fd9b22d60181 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:41.153263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef226dc-d7dc-454d-a1be-fd9b22d60181 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:41.153536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef226dc-d7dc-454d-a1be-fd9b22d60181 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:41.160083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef226dc-d7dc-454d-a1be-fd9b22d60181 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:41.161502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef226dc-d7dc-454d-a1be-fd9b22d60181 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-eef226dc-d7dc-454d-a1be-fd9b22d60181 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:40:41.174351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:41.175487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:41.176656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:40:41.176656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:40:41.176656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:41.176656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:41.176656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:41.176656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:41.176656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:41.176656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:41.176656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 537/1077] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:41.181799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef226dc-d7dc-454d-a1be-fd9b22d60181 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:41.197649 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea6ef2a-0ca1-406b-8cf3-79d2ba5af106 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:41.222358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:41.235112 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9ccf8-0be7-4daa-ab1e-7513cad62f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:41.235746 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9ccf8-0be7-4daa-ab1e-7513cad62f95 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:41.236023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9ccf8-0be7-4daa-ab1e-7513cad62f95 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:41.236294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9ccf8-0be7-4daa-ab1e-7513cad62f95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:41.246865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9ccf8-0be7-4daa-ab1e-7513cad62f95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-08d9ccf8-0be7-4daa-ab1e-7513cad62f95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:41.252144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:41.253100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:41.254010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:41.254010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:41.254010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:41.254010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:41.254010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:41.254010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:41.254010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:41.254010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:41.254010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:41.254010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:41.254010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:41.254010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:41.254010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 541/1078] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:41.259221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9ccf8-0be7-4daa-ab1e-7513cad62f95 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:41.279417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef226dc-d7dc-454d-a1be-fd9b22d60181 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:41.296215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:41.315277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bd0c2-18ed-413c-9c33-deaed663d5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:41.315885 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bd0c2-18ed-413c-9c33-deaed663d5ee None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:41.316164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bd0c2-18ed-413c-9c33-deaed663d5ee None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:41.316436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bd0c2-18ed-413c-9c33-deaed663d5ee None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:41.317123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bd0c2-18ed-413c-9c33-deaed663d5ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:41.317426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bd0c2-18ed-413c-9c33-deaed663d5ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:41.319154 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bd0c2-18ed-413c-9c33-deaed663d5ee None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:41.331005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 538/1079] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:40:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:41.337039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bd0c2-18ed-413c-9c33-deaed663d5ee None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:41.356937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9ccf8-0be7-4daa-ab1e-7513cad62f95 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:41.390914 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c497062-99ff-4c7f-8f2d-45426a7ca7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:41.391512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c497062-99ff-4c7f-8f2d-45426a7ca7fa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:41.391791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c497062-99ff-4c7f-8f2d-45426a7ca7fa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:41.392057 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c497062-99ff-4c7f-8f2d-45426a7ca7fa None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:41.392757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c497062-99ff-4c7f-8f2d-45426a7ca7fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:41.393054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c497062-99ff-4c7f-8f2d-45426a7ca7fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:41.394431 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c497062-99ff-4c7f-8f2d-45426a7ca7fa None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:41.403191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 542/1080] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:41.408839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c497062-99ff-4c7f-8f2d-45426a7ca7fa None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:41.436037 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bd0c2-18ed-413c-9c33-deaed663d5ee None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:41.459199 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:41.471388 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd0e88-8900-4fc1-a4ce-c3f910773ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:41.472066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd0e88-8900-4fc1-a4ce-c3f910773ee4 None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:41.472343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd0e88-8900-4fc1-a4ce-c3f910773ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:41.472657 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd0e88-8900-4fc1-a4ce-c3f910773ee4 None admin] PATH_INFO: `/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/6ffa73f81539472391983c51447af1b6/roles/a50724aa8b014431acb0d653bf1e41e7` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:41.474766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd0e88-8900-4fc1-a4ce-c3f910773ee4 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe9c4e7bc14e70b57bf834cfff365f, user_id=6ffa73f81539472391983c51447af1b6, role_id=a50724aa8b014431acb0d653bf1e41e7)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:41.476188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd0e88-8900-4fc1-a4ce-c3f910773ee4 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:41.489205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 539/1081] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:40:41 2022] PUT /identity/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/6ffa73f81539472391983c51447af1b6/roles/a50724aa8b014431acb0d653bf1e41e7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:40:41.519681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c497062-99ff-4c7f-8f2d-45426a7ca7fa None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:41.557527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0aaaf7-1910-4128-acd1-e0be8cfdceed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:41.558209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aaaf7-1910-4128-acd1-e0be8cfdceed None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:41.558487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aaaf7-1910-4128-acd1-e0be8cfdceed None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:41.558760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0aaaf7-1910-4128-acd1-e0be8cfdceed None admin] PATH_INFO: `/v3/users/313377cf08c54e959473499b18db7acc` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:41.559969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0aaaf7-1910-4128-acd1-e0be8cfdceed None admin] RBAC: Authorizing `identity:get_user(user_id=313377cf08c54e959473499b18db7acc)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:41.561421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0aaaf7-1910-4128-acd1-e0be8cfdceed None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:41.563093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 543/1082] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:41 2022] GET /identity/v3/users/313377cf08c54e959473499b18db7acc => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:41.569413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd0e88-8900-4fc1-a4ce-c3f910773ee4 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:41.607004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:41.677922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd0e88-8900-4fc1-a4ce-c3f910773ee4 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:41.712012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f262c-a005-47f2-b3b6-e0d8f678aa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:41.712634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f262c-a005-47f2-b3b6-e0d8f678aa51 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:41.712912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f262c-a005-47f2-b3b6-e0d8f678aa51 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:41.713183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f262c-a005-47f2-b3b6-e0d8f678aa51 None admin] PATH_INFO: `/v3/projects/ae8123a373ac45a3914597a54687e842` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:41.714324 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f262c-a005-47f2-b3b6-e0d8f678aa51 None admin] RBAC: Authorizing `identity:get_project(project_id=ae8123a373ac45a3914597a54687e842)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:41.715733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f262c-a005-47f2-b3b6-e0d8f678aa51 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:41.717356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 540/1083] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:41 2022] GET /identity/v3/projects/ae8123a373ac45a3914597a54687e842 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:41.723407 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0aaaf7-1910-4128-acd1-e0be8cfdceed None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:41.821305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0aaaf7-1910-4128-acd1-e0be8cfdceed None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:41.837264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:41.855638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d9e03-d405-43ce-bd86-80d15bf5a9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:41.856299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d9e03-d405-43ce-bd86-80d15bf5a9b5 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:41.856578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d9e03-d405-43ce-bd86-80d15bf5a9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:41.856849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d9e03-d405-43ce-bd86-80d15bf5a9b5 None admin] PATH_INFO: `/v3/projects/ae8123a373ac45a3914597a54687e842/users/313377cf08c54e959473499b18db7acc/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:41.858774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d9e03-d405-43ce-bd86-80d15bf5a9b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae8123a373ac45a3914597a54687e842, user_id=313377cf08c54e959473499b18db7acc, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:41.860199 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d9e03-d405-43ce-bd86-80d15bf5a9b5 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:41.873446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 544/1084] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:40:41 2022] PUT /identity/v3/projects/ae8123a373ac45a3914597a54687e842/users/313377cf08c54e959473499b18db7acc/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:40:43.616034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392b203-18fc-41be-9e96-44484ee1ab48 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:43.616599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392b203-18fc-41be-9e96-44484ee1ab48 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:43.616599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392b203-18fc-41be-9e96-44484ee1ab48 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:43.618468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 541/1085] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:43.623658 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a8f0b-d57d-4993-962c-a3cdb6da8490 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:43.624056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a8f0b-d57d-4993-962c-a3cdb6da8490 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:43.624354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a8f0b-d57d-4993-962c-a3cdb6da8490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:43.642458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-212a8f0b-d57d-4993-962c-a3cdb6da8490 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:43.643636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-212a8f0b-d57d-4993-962c-a3cdb6da8490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:43.708312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 545/1086] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:43.984837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392b203-18fc-41be-9e96-44484ee1ab48 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:43.988196 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a8c7-8b57-49aa-95ff-096840f04451 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:43.988488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a8c7-8b57-49aa-95ff-096840f04451 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:43.988757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a8c7-8b57-49aa-95ff-096840f04451 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:43.990200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 546/1087] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:43.995685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21358-1d54-4c8d-9df8-093f19667008 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:43.995963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21358-1d54-4c8d-9df8-093f19667008 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:43.996233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb21358-1d54-4c8d-9df8-093f19667008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:44.014544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ddb21358-1d54-4c8d-9df8-093f19667008 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:44.015733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ddb21358-1d54-4c8d-9df8-093f19667008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:44.080515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 547/1088] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:44.086778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392b203-18fc-41be-9e96-44484ee1ab48 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:44.089051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:44.120469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027b3b9-39cc-4536-a7b3-9d383da78a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:44.121087 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027b3b9-39cc-4536-a7b3-9d383da78a3b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:44.121365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027b3b9-39cc-4536-a7b3-9d383da78a3b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:44.121637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027b3b9-39cc-4536-a7b3-9d383da78a3b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:44.135765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b027b3b9-39cc-4536-a7b3-9d383da78a3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-b027b3b9-39cc-4536-a7b3-9d383da78a3b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:44.142143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:44.143292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:44.144216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:44.144216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:44.144216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:44.144216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:44.144216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:44.144216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:44.144216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:44.144216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:44.144216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:44.144216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:44.144216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:44.144216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:44.144216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 542/1089] 10.222.0.55 () {56 vars in 1120 bytes} [Thu Sep 8 13:40:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:44.150351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21358-1d54-4c8d-9df8-093f19667008 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:44.183641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:44.246771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb21358-1d54-4c8d-9df8-093f19667008 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:44.280313 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664e543-10a4-4384-94e5-464537cbc42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:44.280919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664e543-10a4-4384-94e5-464537cbc42a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:44.281197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664e543-10a4-4384-94e5-464537cbc42a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:44.281469 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664e543-10a4-4384-94e5-464537cbc42a None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:44.282163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664e543-10a4-4384-94e5-464537cbc42a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:44.282469 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664e543-10a4-4384-94e5-464537cbc42a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:44.283822 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664e543-10a4-4384-94e5-464537cbc42a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:44.291382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 548/1090] 10.222.0.55 () {56 vars in 1115 bytes} [Thu Sep 8 13:40:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:44.297168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027b3b9-39cc-4536-a7b3-9d383da78a3b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:44.353615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664e543-10a4-4384-94e5-464537cbc42a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:44.397347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027b3b9-39cc-4536-a7b3-9d383da78a3b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:44.431586 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7f1a4-654a-4316-8c84-3457f0229bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:44.432375 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7f1a4-654a-4316-8c84-3457f0229bac None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:44.432797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7f1a4-654a-4316-8c84-3457f0229bac None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:44.433212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7f1a4-654a-4316-8c84-3457f0229bac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:44.446739 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a7f1a4-654a-4316-8c84-3457f0229bac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:44.448489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a7f1a4-654a-4316-8c84-3457f0229bac None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:44.452005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:44.456278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664e543-10a4-4384-94e5-464537cbc42a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-10a7f1a4-654a-4316-8c84-3457f0229bac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:44.464619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:44.465610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:44.466561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:44.466561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:44.466561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:44.466561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 543/1091] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:44.471378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7f1a4-654a-4316-8c84-3457f0229bac None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:44.490316 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:44.493008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ebcd4-c55e-447a-8663-57a9ab252600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:44.493640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ebcd4-c55e-447a-8663-57a9ab252600 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:44.493954 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ebcd4-c55e-447a-8663-57a9ab252600 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:44.494230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ebcd4-c55e-447a-8663-57a9ab252600 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:44.505086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ebcd4-c55e-447a-8663-57a9ab252600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-a38ebcd4-c55e-447a-8663-57a9ab252600 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:44.509854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:44.510814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:44.511657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:44.511657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:44.511657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:44.511657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:44.511657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:44.511657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:44.511657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:44.511657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:44.511657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:44.511657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:44.511657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:44.511657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:40:44.511657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 549/1092] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:44.517074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ebcd4-c55e-447a-8663-57a9ab252600 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:44.575750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7f1a4-654a-4316-8c84-3457f0229bac None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:44.609025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206d91b-d431-4a3c-9ccb-b045e0cb71c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:44.609620 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206d91b-d431-4a3c-9ccb-b045e0cb71c4 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:44.609921 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206d91b-d431-4a3c-9ccb-b045e0cb71c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:44.610190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206d91b-d431-4a3c-9ccb-b045e0cb71c4 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:44.610819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206d91b-d431-4a3c-9ccb-b045e0cb71c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:44.611119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206d91b-d431-4a3c-9ccb-b045e0cb71c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:44.612460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206d91b-d431-4a3c-9ccb-b045e0cb71c4 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:44.622896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ebcd4-c55e-447a-8663-57a9ab252600 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:44.624145 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 544/1093] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:44.629704 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206d91b-d431-4a3c-9ccb-b045e0cb71c4 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:44.656115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc207a49-d447-4219-bdbf-ffd93d28e160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:44.656723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc207a49-d447-4219-bdbf-ffd93d28e160 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:44.657000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc207a49-d447-4219-bdbf-ffd93d28e160 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:44.657274 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc207a49-d447-4219-bdbf-ffd93d28e160 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:44.657961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc207a49-d447-4219-bdbf-ffd93d28e160 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:44.658271 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc207a49-d447-4219-bdbf-ffd93d28e160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:44.659631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc207a49-d447-4219-bdbf-ffd93d28e160 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:44.670607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 550/1094] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:44.676273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc207a49-d447-4219-bdbf-ffd93d28e160 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:44.697570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:44.711806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:44.735637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206d91b-d431-4a3c-9ccb-b045e0cb71c4 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:44.768979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015d562-03a8-4c74-adab-29bf9fcec098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:44.769601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015d562-03a8-4c74-adab-29bf9fcec098 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:44.769922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015d562-03a8-4c74-adab-29bf9fcec098 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:44.770190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015d562-03a8-4c74-adab-29bf9fcec098 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:44.776159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc207a49-d447-4219-bdbf-ffd93d28e160 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:44.777837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015d562-03a8-4c74-adab-29bf9fcec098 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:44.779993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015d562-03a8-4c74-adab-29bf9fcec098 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-4015d562-03a8-4c74-adab-29bf9fcec098 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:40:44.792450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:44.793535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:44.795032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:44.795032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:44.795032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:40:44.795032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:40:44.795032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:40:44.795032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:40:44.795032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:40:44.795032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:40:44.795032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:40:44.795032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:44.795032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 545/1095] 10.222.0.55 () {56 vars in 1093 bytes} [Thu Sep 8 13:40:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:44.799136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015d562-03a8-4c74-adab-29bf9fcec098 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:44.809269 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dcf4a-2b67-440a-bb23-71c65cf63802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:44.809930 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dcf4a-2b67-440a-bb23-71c65cf63802 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:44.810210 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dcf4a-2b67-440a-bb23-71c65cf63802 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:44.810480 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03dcf4a-2b67-440a-bb23-71c65cf63802 None admin] PATH_INFO: `/v3/users/313377cf08c54e959473499b18db7acc` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:44.811624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03dcf4a-2b67-440a-bb23-71c65cf63802 None admin] RBAC: Authorizing `identity:get_user(user_id=313377cf08c54e959473499b18db7acc)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:44.813040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03dcf4a-2b67-440a-bb23-71c65cf63802 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:44.814638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 551/1096] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:44 2022] GET /identity/v3/users/313377cf08c54e959473499b18db7acc => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:44.820296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dcf4a-2b67-440a-bb23-71c65cf63802 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:44.903864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015d562-03a8-4c74-adab-29bf9fcec098 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:44.907685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:44.921323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03dcf4a-2b67-440a-bb23-71c65cf63802 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:44.938157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bc96c-46db-436d-99f4-d4a9134ccbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:44.939039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bc96c-46db-436d-99f4-d4a9134ccbc9 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:44.939337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bc96c-46db-436d-99f4-d4a9134ccbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:44.939608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bc96c-46db-436d-99f4-d4a9134ccbc9 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:44.940329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bc96c-46db-436d-99f4-d4a9134ccbc9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:44.940629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bc96c-46db-436d-99f4-d4a9134ccbc9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:44.942054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bc96c-46db-436d-99f4-d4a9134ccbc9 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:44.944840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:44.956220 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc833f4-e0af-40fa-9c21-8f4168feaeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:44.956755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc833f4-e0af-40fa-9c21-8f4168feaeb8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:44.957028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc833f4-e0af-40fa-9c21-8f4168feaeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:44.957296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc833f4-e0af-40fa-9c21-8f4168feaeb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:44.962091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 546/1097] 10.222.0.55 () {56 vars in 1133 bytes} [Thu Sep 8 13:40:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:44.968189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bc96c-46db-436d-99f4-d4a9134ccbc9 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:45.022609 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc833f4-e0af-40fa-9c21-8f4168feaeb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:45.024304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc833f4-e0af-40fa-9c21-8f4168feaeb8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:45.072421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bc96c-46db-436d-99f4-d4a9134ccbc9 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:45.105591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb9cc0-d049-4f05-bcf5-7f1296bb3b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:45.106220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb9cc0-d049-4f05-bcf5-7f1296bb3b05 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:45.106499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb9cc0-d049-4f05-bcf5-7f1296bb3b05 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:45.106771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb9cc0-d049-4f05-bcf5-7f1296bb3b05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:45.112163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:45.130263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 552/1098] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:40:45.131233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc833f4-e0af-40fa-9c21-8f4168feaeb8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:45.167993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:45.175480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb9cc0-d049-4f05-bcf5-7f1296bb3b05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:45.176927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb9cc0-d049-4f05-bcf5-7f1296bb3b05 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:45.228040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc833f4-e0af-40fa-9c21-8f4168feaeb8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:45.261717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16a7f0-c174-4da0-8b39-fcba76e9a01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:45.262379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16a7f0-c174-4da0-8b39-fcba76e9a01b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:45.262660 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16a7f0-c174-4da0-8b39-fcba76e9a01b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:45.263042 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16a7f0-c174-4da0-8b39-fcba76e9a01b None admin] PATH_INFO: `/v3/projects/ae8123a373ac45a3914597a54687e842` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:45.264167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16a7f0-c174-4da0-8b39-fcba76e9a01b None admin] RBAC: Authorizing `identity:get_project(project_id=ae8123a373ac45a3914597a54687e842)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:45.265592 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16a7f0-c174-4da0-8b39-fcba76e9a01b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:45.267248 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 553/1099] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:45 2022] GET /identity/v3/projects/ae8123a373ac45a3914597a54687e842 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:45.273218 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16a7f0-c174-4da0-8b39-fcba76e9a01b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:45.277768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 547/1100] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:40:45.279998 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb9cc0-d049-4f05-bcf5-7f1296bb3b05 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:45.328205 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:45.371102 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16a7f0-c174-4da0-8b39-fcba76e9a01b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:45.371323 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:45.377921 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb9cc0-d049-4f05-bcf5-7f1296bb3b05 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:45.405576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225741f1-c24e-49a6-9654-3a9ec4f5872e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:45.406208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225741f1-c24e-49a6-9654-3a9ec4f5872e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:45.406488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225741f1-c24e-49a6-9654-3a9ec4f5872e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:45.406757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225741f1-c24e-49a6-9654-3a9ec4f5872e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:45.407475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225741f1-c24e-49a6-9654-3a9ec4f5872e None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=ae8123a373ac45a3914597a54687e842, user.id=313377cf08c54e959473499b18db7acc) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:45.407783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225741f1-c24e-49a6-9654-3a9ec4f5872e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:45.409200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225741f1-c24e-49a6-9654-3a9ec4f5872e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:45.411541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e2fc6-42e7-4d1d-9afa-2f59a6c22bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:45.412150 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e2fc6-42e7-4d1d-9afa-2f59a6c22bac None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:45.412420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e2fc6-42e7-4d1d-9afa-2f59a6c22bac None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:45.412683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e2fc6-42e7-4d1d-9afa-2f59a6c22bac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:45.414838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 554/1101] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:45 2022] GET /identity/v3/role_assignments?user.id=313377cf08c54e959473499b18db7acc&scope.project.id=ae8123a373ac45a3914597a54687e842&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:45.427330 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e2fc6-42e7-4d1d-9afa-2f59a6c22bac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:45.428817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e2fc6-42e7-4d1d-9afa-2f59a6c22bac None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-f22e2fc6-42e7-4d1d-9afa-2f59a6c22bac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:45.446512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:45.447758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:45.449147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:45.449147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:45.449147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:45.449147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 548/1102] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:45.453647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225741f1-c24e-49a6-9654-3a9ec4f5872e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:45.568234 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225741f1-c24e-49a6-9654-3a9ec4f5872e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:45.589543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:45.606241 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170600ad-fe45-473f-935e-ef81fef8fc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:45.606860 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170600ad-fe45-473f-935e-ef81fef8fc5b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:45.607142 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170600ad-fe45-473f-935e-ef81fef8fc5b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:45.607417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170600ad-fe45-473f-935e-ef81fef8fc5b None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:45.608062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170600ad-fe45-473f-935e-ef81fef8fc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:45.608370 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170600ad-fe45-473f-935e-ef81fef8fc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:45.609774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170600ad-fe45-473f-935e-ef81fef8fc5b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:45.622165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 555/1103] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:45.628149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e2fc6-42e7-4d1d-9afa-2f59a6c22bac None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:45.724674 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e2fc6-42e7-4d1d-9afa-2f59a6c22bac None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:45.757327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95373e8d-edd8-4be7-9115-c8b4a0a9bbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:45.757964 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95373e8d-edd8-4be7-9115-c8b4a0a9bbc0 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:45.758757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95373e8d-edd8-4be7-9115-c8b4a0a9bbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:45.759027 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95373e8d-edd8-4be7-9115-c8b4a0a9bbc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:45.765465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95373e8d-edd8-4be7-9115-c8b4a0a9bbc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:45.766919 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95373e8d-edd8-4be7-9115-c8b4a0a9bbc0 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-95373e8d-edd8-4be7-9115-c8b4a0a9bbc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:40:45.778211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:45.779306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:45.780436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:40:45.780436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:40:45.780436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:45.780436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:45.780436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:45.780436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:45.780436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:45.780436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:40:45.780436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 549/1104] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:45.785237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170600ad-fe45-473f-935e-ef81fef8fc5b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:45.885028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170600ad-fe45-473f-935e-ef81fef8fc5b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:45.921660 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b352da-7a32-48fe-bcd1-ae2b2ede621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:45.922312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b352da-7a32-48fe-bcd1-ae2b2ede621e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:45.922587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b352da-7a32-48fe-bcd1-ae2b2ede621e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:45.922856 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b352da-7a32-48fe-bcd1-ae2b2ede621e None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:45.923535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b352da-7a32-48fe-bcd1-ae2b2ede621e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:45.923840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b352da-7a32-48fe-bcd1-ae2b2ede621e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:45.925257 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b352da-7a32-48fe-bcd1-ae2b2ede621e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:45.936567 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 556/1105] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:40:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:45.942554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95373e8d-edd8-4be7-9115-c8b4a0a9bbc0 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:45.952106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:46.042389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95373e8d-edd8-4be7-9115-c8b4a0a9bbc0 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:46.075066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e40ab-89d5-4f61-8813-0b65f496f8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:46.075653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e40ab-89d5-4f61-8813-0b65f496f8b8 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:46.075928 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e40ab-89d5-4f61-8813-0b65f496f8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:46.076197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e40ab-89d5-4f61-8813-0b65f496f8b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:46.076912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e40ab-89d5-4f61-8813-0b65f496f8b8 None admin] RBAC: Adding query filter params (role.id=a50724aa8b014431acb0d653bf1e41e7, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=6ffa73f81539472391983c51447af1b6) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:46.077215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e40ab-89d5-4f61-8813-0b65f496f8b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:46.078661 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e40ab-89d5-4f61-8813-0b65f496f8b8 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:46.084502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 550/1106] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:45 2022] GET /identity/v3/role_assignments?user.id=6ffa73f81539472391983c51447af1b6&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=a50724aa8b014431acb0d653bf1e41e7 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:47.589988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c545d0b-f12e-445d-82a2-6096f488a074 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:47.590517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c545d0b-f12e-445d-82a2-6096f488a074 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:47.590517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c545d0b-f12e-445d-82a2-6096f488a074 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:47.591856 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 557/1107] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:47.596938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f55432-a03d-449c-804d-7cea0bbbdcb4 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:47.597370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f55432-a03d-449c-804d-7cea0bbbdcb4 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:47.597478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f55432-a03d-449c-804d-7cea0bbbdcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:47.615917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-27f55432-a03d-449c-804d-7cea0bbbdcb4 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:47.617109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-27f55432-a03d-449c-804d-7cea0bbbdcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:47.677275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:47.682316 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 551/1108] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:47.948376 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c545d0b-f12e-445d-82a2-6096f488a074 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:47.989590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:48.049814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c545d0b-f12e-445d-82a2-6096f488a074 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:48.096134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6353407-8029-49d0-9035-ff973e2cce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:48.097103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6353407-8029-49d0-9035-ff973e2cce16 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:48.097570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6353407-8029-49d0-9035-ff973e2cce16 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:48.098058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6353407-8029-49d0-9035-ff973e2cce16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:48.099804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6353407-8029-49d0-9035-ff973e2cce16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:48.102306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6353407-8029-49d0-9035-ff973e2cce16 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:48.105091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 558/1109] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:48.110853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f55432-a03d-449c-804d-7cea0bbbdcb4 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:48.208367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f55432-a03d-449c-804d-7cea0bbbdcb4 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:48.240847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9031f2-bf32-4068-b0c9-c8a58ab8f49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:48.241449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9031f2-bf32-4068-b0c9-c8a58ab8f49c None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:48.241756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9031f2-bf32-4068-b0c9-c8a58ab8f49c None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:48.242031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9031f2-bf32-4068-b0c9-c8a58ab8f49c None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:48.243214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9031f2-bf32-4068-b0c9-c8a58ab8f49c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:48.244642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9031f2-bf32-4068-b0c9-c8a58ab8f49c None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:48.272675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 552/1110] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:40:48 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:50.442243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4007477-5d37-44fa-9f0c-996d70640dde None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:50.443167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4007477-5d37-44fa-9f0c-996d70640dde None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:50.443167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4007477-5d37-44fa-9f0c-996d70640dde None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:50.445025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 559/1111] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:50.450316 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a2053-f3ef-4080-9781-710be19516a1 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:50.450752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a2053-f3ef-4080-9781-710be19516a1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:50.450859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a2053-f3ef-4080-9781-710be19516a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:50.469136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-085a2053-f3ef-4080-9781-710be19516a1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:50.470355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-085a2053-f3ef-4080-9781-710be19516a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:50.519509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:50.536842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 553/1112] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:50.812935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4007477-5d37-44fa-9f0c-996d70640dde None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:50.910303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4007477-5d37-44fa-9f0c-996d70640dde None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:50.919249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:50.943583 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d45fc2-69da-4100-a7e6-ccfcf9de5e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:50.944207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d45fc2-69da-4100-a7e6-ccfcf9de5e2c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:50.944487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d45fc2-69da-4100-a7e6-ccfcf9de5e2c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:50.944763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d45fc2-69da-4100-a7e6-ccfcf9de5e2c None admin] PATH_INFO: `/v3/roles/7d0ac07c6114438da47dcfefeb1fe05b` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:50.946660 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d45fc2-69da-4100-a7e6-ccfcf9de5e2c None admin] RBAC: Authorizing `identity:get_role(role_id=7d0ac07c6114438da47dcfefeb1fe05b)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:50.948030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d45fc2-69da-4100-a7e6-ccfcf9de5e2c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:50.949377 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 560/1113] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:50 2022] GET /identity/v3/roles/7d0ac07c6114438da47dcfefeb1fe05b => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:50.955148 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a2053-f3ef-4080-9781-710be19516a1 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:51.054209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a2053-f3ef-4080-9781-710be19516a1 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:51.087258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a43793-2036-449e-b401-313668c20bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:51.087808 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a43793-2036-449e-b401-313668c20bae None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:51.088174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a43793-2036-449e-b401-313668c20bae None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:51.088444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a43793-2036-449e-b401-313668c20bae None admin] PATH_INFO: `/v3/users/f513cbcdfdac4df2987e0c826dd3e37d` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:51.102937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a43793-2036-449e-b401-313668c20bae None admin] RBAC: Authorizing `identity:get_user(user_id=f513cbcdfdac4df2987e0c826dd3e37d)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:51.104357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a43793-2036-449e-b401-313668c20bae None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:51.105965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 554/1114] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:50 2022] GET /identity/v3/users/f513cbcdfdac4df2987e0c826dd3e37d => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:51.111788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d45fc2-69da-4100-a7e6-ccfcf9de5e2c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:51.208678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d45fc2-69da-4100-a7e6-ccfcf9de5e2c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:51.240739 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0a98f-ce43-4199-b2d3-6ee7e40a8767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:51.241334 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0a98f-ce43-4199-b2d3-6ee7e40a8767 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:51.241613 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0a98f-ce43-4199-b2d3-6ee7e40a8767 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:51.241913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0a98f-ce43-4199-b2d3-6ee7e40a8767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:51.285186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:51.310551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0a98f-ce43-4199-b2d3-6ee7e40a8767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:51.311998 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b0a98f-ce43-4199-b2d3-6ee7e40a8767 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:51.407129 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 561/1115] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:51.413045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a43793-2036-449e-b401-313668c20bae None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:51.457625 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:51.509778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a43793-2036-449e-b401-313668c20bae None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:51.541907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496967fe-8e5b-47e5-beee-acc85f5cf4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:51.542515 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496967fe-8e5b-47e5-beee-acc85f5cf4df None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:51.542790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496967fe-8e5b-47e5-beee-acc85f5cf4df None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:51.543058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496967fe-8e5b-47e5-beee-acc85f5cf4df None admin] PATH_INFO: `/v3/projects/ae8123a373ac45a3914597a54687e842` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:51.544152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496967fe-8e5b-47e5-beee-acc85f5cf4df None admin] RBAC: Authorizing `identity:get_project(project_id=ae8123a373ac45a3914597a54687e842)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:51.545554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496967fe-8e5b-47e5-beee-acc85f5cf4df None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:51.547173 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 555/1116] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:51 2022] GET /identity/v3/projects/ae8123a373ac45a3914597a54687e842 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:51.553308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0a98f-ce43-4199-b2d3-6ee7e40a8767 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:51.631335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:51.655653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0a98f-ce43-4199-b2d3-6ee7e40a8767 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:51.689761 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca063ab0-c9a1-466b-aa8d-658cc213bcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:51.690295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca063ab0-c9a1-466b-aa8d-658cc213bcfb None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:51.690575 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca063ab0-c9a1-466b-aa8d-658cc213bcfb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:51.690850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca063ab0-c9a1-466b-aa8d-658cc213bcfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:51.691561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca063ab0-c9a1-466b-aa8d-658cc213bcfb None admin] RBAC: Adding query filter params (role.id=7d0ac07c6114438da47dcfefeb1fe05b, scope.project.id=ae8123a373ac45a3914597a54687e842, user.id=f513cbcdfdac4df2987e0c826dd3e37d) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:51.691866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca063ab0-c9a1-466b-aa8d-658cc213bcfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:51.693538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca063ab0-c9a1-466b-aa8d-658cc213bcfb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:51.698763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 562/1117] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:51 2022] GET /identity/v3/role_assignments?user.id=f513cbcdfdac4df2987e0c826dd3e37d&scope.project.id=ae8123a373ac45a3914597a54687e842&role.id=7d0ac07c6114438da47dcfefeb1fe05b => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:53.830750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0db87-9f36-49b9-9fee-6de6cbdb9097 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:53.831342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0db87-9f36-49b9-9fee-6de6cbdb9097 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:53.831342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0db87-9f36-49b9-9fee-6de6cbdb9097 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:53.832637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 556/1118] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:53.837753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0a6e9-ebd1-4a6d-9ffb-75b5385eeb2e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:53.838124 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0a6e9-ebd1-4a6d-9ffb-75b5385eeb2e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:53.838305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0a6e9-ebd1-4a6d-9ffb-75b5385eeb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:53.856291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-dab0a6e9-ebd1-4a6d-9ffb-75b5385eeb2e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:53.857441 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dab0a6e9-ebd1-4a6d-9ffb-75b5385eeb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:53.923449 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 563/1119] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:54.191171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0db87-9f36-49b9-9fee-6de6cbdb9097 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:54.290847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0db87-9f36-49b9-9fee-6de6cbdb9097 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:54.303330 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:54.323820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48914502-ca7a-4307-995b-e01c37f8d469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:54.324431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48914502-ca7a-4307-995b-e01c37f8d469 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:54.324709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48914502-ca7a-4307-995b-e01c37f8d469 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:54.324978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48914502-ca7a-4307-995b-e01c37f8d469 None admin] PATH_INFO: `/v3/roles/7d0ac07c6114438da47dcfefeb1fe05b` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:54.326455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48914502-ca7a-4307-995b-e01c37f8d469 None admin] RBAC: Authorizing `identity:get_role(role_id=7d0ac07c6114438da47dcfefeb1fe05b)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:54.327812 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48914502-ca7a-4307-995b-e01c37f8d469 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:54.329111 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 557/1120] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:54 2022] GET /identity/v3/roles/7d0ac07c6114438da47dcfefeb1fe05b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:54.334823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0a6e9-ebd1-4a6d-9ffb-75b5385eeb2e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:54.400221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:54.429725 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0a6e9-ebd1-4a6d-9ffb-75b5385eeb2e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:54.462247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e69da1-5420-4a1a-b49b-3aa1f8b51201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:54.462802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e69da1-5420-4a1a-b49b-3aa1f8b51201 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:54.463075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e69da1-5420-4a1a-b49b-3aa1f8b51201 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:54.463344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e69da1-5420-4a1a-b49b-3aa1f8b51201 None admin] PATH_INFO: `/v3/users/f513cbcdfdac4df2987e0c826dd3e37d` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:54.464434 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e69da1-5420-4a1a-b49b-3aa1f8b51201 None admin] RBAC: Authorizing `identity:get_user(user_id=f513cbcdfdac4df2987e0c826dd3e37d)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:54.465879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e69da1-5420-4a1a-b49b-3aa1f8b51201 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:54.467411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 564/1121] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:54 2022] GET /identity/v3/users/f513cbcdfdac4df2987e0c826dd3e37d => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:54.473010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48914502-ca7a-4307-995b-e01c37f8d469 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:54.569990 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48914502-ca7a-4307-995b-e01c37f8d469 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:54.603440 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b1917-41ef-4ee2-b66a-8b099113f421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:54.604051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b1917-41ef-4ee2-b66a-8b099113f421 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:54.604332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b1917-41ef-4ee2-b66a-8b099113f421 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:54.604607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b1917-41ef-4ee2-b66a-8b099113f421 None admin] PATH_INFO: `/v3/projects/ae8123a373ac45a3914597a54687e842` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:54.605713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b1917-41ef-4ee2-b66a-8b099113f421 None admin] RBAC: Authorizing `identity:get_project(project_id=ae8123a373ac45a3914597a54687e842)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:54.607151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b1917-41ef-4ee2-b66a-8b099113f421 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:54.608747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 558/1122] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:54 2022] GET /identity/v3/projects/ae8123a373ac45a3914597a54687e842 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:54.614591 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e69da1-5420-4a1a-b49b-3aa1f8b51201 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:54.710802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e69da1-5420-4a1a-b49b-3aa1f8b51201 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:54.743318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4823c-f08e-4cb2-b8d0-ded6cad91e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:54.743897 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4823c-f08e-4cb2-b8d0-ded6cad91e66 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:54.744170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4823c-f08e-4cb2-b8d0-ded6cad91e66 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:54.744437 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4823c-f08e-4cb2-b8d0-ded6cad91e66 None admin] PATH_INFO: `/v3/projects/ae8123a373ac45a3914597a54687e842/users/f513cbcdfdac4df2987e0c826dd3e37d/roles/7d0ac07c6114438da47dcfefeb1fe05b` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:54.746519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4823c-f08e-4cb2-b8d0-ded6cad91e66 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae8123a373ac45a3914597a54687e842, user_id=f513cbcdfdac4df2987e0c826dd3e37d, role_id=7d0ac07c6114438da47dcfefeb1fe05b)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:54.747928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4823c-f08e-4cb2-b8d0-ded6cad91e66 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:54.755551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:54.762057 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 565/1123] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:40:54 2022] PUT /identity/v3/projects/ae8123a373ac45a3914597a54687e842/users/f513cbcdfdac4df2987e0c826dd3e37d/roles/7d0ac07c6114438da47dcfefeb1fe05b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:40:56.877430 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6654368-6ef9-4868-8fd9-1389de525f22 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:56.877984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6654368-6ef9-4868-8fd9-1389de525f22 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:56.877984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6654368-6ef9-4868-8fd9-1389de525f22 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:56.879325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 559/1124] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:56.884619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40174d6-0bee-4c74-b8e2-6df9a8f2896c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:56.885041 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40174d6-0bee-4c74-b8e2-6df9a8f2896c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:56.885174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40174d6-0bee-4c74-b8e2-6df9a8f2896c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:56.903052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d40174d6-0bee-4c74-b8e2-6df9a8f2896c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:56.904267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d40174d6-0bee-4c74-b8e2-6df9a8f2896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:56.968589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 566/1125] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:57.238773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6654368-6ef9-4868-8fd9-1389de525f22 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:57.286829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:57.336444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6654368-6ef9-4868-8fd9-1389de525f22 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:57.368772 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa6636-9a5a-4750-9108-d9e5e6d68fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:57.369374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6636-9a5a-4750-9108-d9e5e6d68fd5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:57.369653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6636-9a5a-4750-9108-d9e5e6d68fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:57.369961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6636-9a5a-4750-9108-d9e5e6d68fd5 None admin] PATH_INFO: `/v3/roles/7d0ac07c6114438da47dcfefeb1fe05b` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:57.371330 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa6636-9a5a-4750-9108-d9e5e6d68fd5 None admin] RBAC: Authorizing `identity:get_role(role_id=7d0ac07c6114438da47dcfefeb1fe05b)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:57.372685 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa6636-9a5a-4750-9108-d9e5e6d68fd5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:57.374017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 560/1126] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:57 2022] GET /identity/v3/roles/7d0ac07c6114438da47dcfefeb1fe05b => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:57.379682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40174d6-0bee-4c74-b8e2-6df9a8f2896c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:57.474105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40174d6-0bee-4c74-b8e2-6df9a8f2896c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:57.499944 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:57.508227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ddf90-6d89-4ce6-97f2-61e97bb7289c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:57.508850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ddf90-6d89-4ce6-97f2-61e97bb7289c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:57.509129 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ddf90-6d89-4ce6-97f2-61e97bb7289c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:57.509403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ddf90-6d89-4ce6-97f2-61e97bb7289c None admin] PATH_INFO: `/v3/users/f513cbcdfdac4df2987e0c826dd3e37d` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:57.510550 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ddf90-6d89-4ce6-97f2-61e97bb7289c None admin] RBAC: Authorizing `identity:get_user(user_id=f513cbcdfdac4df2987e0c826dd3e37d)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:57.511973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ddf90-6d89-4ce6-97f2-61e97bb7289c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:57.513528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 567/1127] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:57 2022] GET /identity/v3/users/f513cbcdfdac4df2987e0c826dd3e37d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:57.519200 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa6636-9a5a-4750-9108-d9e5e6d68fd5 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:57.615263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa6636-9a5a-4750-9108-d9e5e6d68fd5 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:57.630849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:57.649008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846812a5-b50a-4ec5-b0e7-d0d5f0bd2e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:57.649596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846812a5-b50a-4ec5-b0e7-d0d5f0bd2e9b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:57.649905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846812a5-b50a-4ec5-b0e7-d0d5f0bd2e9b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:57.650174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846812a5-b50a-4ec5-b0e7-d0d5f0bd2e9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:57.714050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846812a5-b50a-4ec5-b0e7-d0d5f0bd2e9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:57.715474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846812a5-b50a-4ec5-b0e7-d0d5f0bd2e9b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:57.808014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 561/1128] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:57.813984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ddf90-6d89-4ce6-97f2-61e97bb7289c None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:57.912840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ddf90-6d89-4ce6-97f2-61e97bb7289c None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:57.947140 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1192f96-402f-4e19-ba1a-ecacf09c1a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:57.947758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1192f96-402f-4e19-ba1a-ecacf09c1a5c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:57.948036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1192f96-402f-4e19-ba1a-ecacf09c1a5c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:57.948310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1192f96-402f-4e19-ba1a-ecacf09c1a5c None admin] PATH_INFO: `/v3/projects/ae8123a373ac45a3914597a54687e842` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:57.949415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1192f96-402f-4e19-ba1a-ecacf09c1a5c None admin] RBAC: Authorizing `identity:get_project(project_id=ae8123a373ac45a3914597a54687e842)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:57.950873 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1192f96-402f-4e19-ba1a-ecacf09c1a5c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:57.952503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 568/1129] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:57 2022] GET /identity/v3/projects/ae8123a373ac45a3914597a54687e842 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:57.958755 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846812a5-b50a-4ec5-b0e7-d0d5f0bd2e9b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:57.976072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:58.060553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846812a5-b50a-4ec5-b0e7-d0d5f0bd2e9b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:58.093020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296431a2-40a5-43be-ae99-e32d355b0f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:58.093612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296431a2-40a5-43be-ae99-e32d355b0f75 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:58.093919 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296431a2-40a5-43be-ae99-e32d355b0f75 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:58.094192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296431a2-40a5-43be-ae99-e32d355b0f75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:58.094906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296431a2-40a5-43be-ae99-e32d355b0f75 None admin] RBAC: Adding query filter params (role.id=7d0ac07c6114438da47dcfefeb1fe05b, scope.project.id=ae8123a373ac45a3914597a54687e842, user.id=f513cbcdfdac4df2987e0c826dd3e37d) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:58.095211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296431a2-40a5-43be-ae99-e32d355b0f75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:58.096618 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296431a2-40a5-43be-ae99-e32d355b0f75 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:58.102701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 562/1130] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:57 2022] GET /identity/v3/role_assignments?user.id=f513cbcdfdac4df2987e0c826dd3e37d&scope.project.id=ae8123a373ac45a3914597a54687e842&role.id=7d0ac07c6114438da47dcfefeb1fe05b => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:41:00.236020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c802e-89b9-49af-beba-a8ef958cf728 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:00.236518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c802e-89b9-49af-beba-a8ef958cf728 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:00.236518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c802e-89b9-49af-beba-a8ef958cf728 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:00.237833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 569/1131] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:00.243184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2a3a5-d40f-4a7f-8bb5-c9a487fc4ea2 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:00.243553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2a3a5-d40f-4a7f-8bb5-c9a487fc4ea2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:00.243735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2a3a5-d40f-4a7f-8bb5-c9a487fc4ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:00.261434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-59c2a3a5-d40f-4a7f-8bb5-c9a487fc4ea2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:00.262647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-59c2a3a5-d40f-4a7f-8bb5-c9a487fc4ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:00.327694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 563/1132] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:00.597919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c802e-89b9-49af-beba-a8ef958cf728 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:00.658518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:00.697849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c802e-89b9-49af-beba-a8ef958cf728 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:00.730405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3764ccd-6384-40c3-b3a0-b5aaad99e99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:00.731016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3764ccd-6384-40c3-b3a0-b5aaad99e99e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:00.731293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3764ccd-6384-40c3-b3a0-b5aaad99e99e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:00.731564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3764ccd-6384-40c3-b3a0-b5aaad99e99e None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:00.732661 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3764ccd-6384-40c3-b3a0-b5aaad99e99e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:00.734104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3764ccd-6384-40c3-b3a0-b5aaad99e99e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:00.735641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 570/1133] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:41:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:00.741517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2a3a5-d40f-4a7f-8bb5-c9a487fc4ea2 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:00.752777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:00.837836 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2a3a5-d40f-4a7f-8bb5-c9a487fc4ea2 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:00.870176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8319993-1af4-4c15-af40-2e51b591fa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:00.870759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8319993-1af4-4c15-af40-2e51b591fa63 None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:00.871035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8319993-1af4-4c15-af40-2e51b591fa63 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:00.871305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8319993-1af4-4c15-af40-2e51b591fa63 None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:00.871947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8319993-1af4-4c15-af40-2e51b591fa63 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:00.873343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8319993-1af4-4c15-af40-2e51b591fa63 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:00.899013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 564/1134] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:41:00 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:41:03.037941 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50065c-ba86-4556-80e6-403862c920e5 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:03.038990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50065c-ba86-4556-80e6-403862c920e5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:03.038990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50065c-ba86-4556-80e6-403862c920e5 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:03.040294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 571/1135] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:03.045640 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194eaa5-ab4b-4e1d-97fa-e5ea6c4236af None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:03.046067 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194eaa5-ab4b-4e1d-97fa-e5ea6c4236af None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:03.046239 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194eaa5-ab4b-4e1d-97fa-e5ea6c4236af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:03.064582 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-8194eaa5-ab4b-4e1d-97fa-e5ea6c4236af None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:03.065778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8194eaa5-ab4b-4e1d-97fa-e5ea6c4236af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:03.121442 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:03.132103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 565/1136] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:03.403479 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50065c-ba86-4556-80e6-403862c920e5 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:03.509101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50065c-ba86-4556-80e6-403862c920e5 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:03.537831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:03.542961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddbec5-fc07-44a3-b3a5-276247f81291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:03.543584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddbec5-fc07-44a3-b3a5-276247f81291 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:03.543863 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddbec5-fc07-44a3-b3a5-276247f81291 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:03.544134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddbec5-fc07-44a3-b3a5-276247f81291 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:03.545235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ddbec5-fc07-44a3-b3a5-276247f81291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:03.546684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ddbec5-fc07-44a3-b3a5-276247f81291 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:03.548227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 572/1137] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:41:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:03.553994 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194eaa5-ab4b-4e1d-97fa-e5ea6c4236af None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:03.649966 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194eaa5-ab4b-4e1d-97fa-e5ea6c4236af None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:03.682217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf865513-58b7-45e7-aecd-f115cb3d0729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:03.682751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf865513-58b7-45e7-aecd-f115cb3d0729 None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:03.683031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf865513-58b7-45e7-aecd-f115cb3d0729 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:03.683304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf865513-58b7-45e7-aecd-f115cb3d0729 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:03.683943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf865513-58b7-45e7-aecd-f115cb3d0729 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:03.685350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf865513-58b7-45e7-aecd-f115cb3d0729 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:03.705823 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 566/1138] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:41:03 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:41:05.869341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffedd9-f3d2-4099-b7a3-8ed6bde40d4c None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:05.869901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffedd9-f3d2-4099-b7a3-8ed6bde40d4c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:05.869901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffedd9-f3d2-4099-b7a3-8ed6bde40d4c None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:05.871240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 573/1139] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:05.876611 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8d693-195c-47f6-8824-8efedb1e4d7d None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:05.877011 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8d693-195c-47f6-8824-8efedb1e4d7d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:05.877153 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac8d693-195c-47f6-8824-8efedb1e4d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:05.895436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-dac8d693-195c-47f6-8824-8efedb1e4d7d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:05.896608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-dac8d693-195c-47f6-8824-8efedb1e4d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:05.939329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:05.964141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 567/1140] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:06.233769 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffedd9-f3d2-4099-b7a3-8ed6bde40d4c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:06.330243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffedd9-f3d2-4099-b7a3-8ed6bde40d4c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:06.363372 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dadeb-c4c9-4bf9-b73b-c269d5bbef37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:06.363983 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dadeb-c4c9-4bf9-b73b-c269d5bbef37 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:06.364259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dadeb-c4c9-4bf9-b73b-c269d5bbef37 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:06.364528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772dadeb-c4c9-4bf9-b73b-c269d5bbef37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:06.374272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772dadeb-c4c9-4bf9-b73b-c269d5bbef37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-772dadeb-c4c9-4bf9-b73b-c269d5bbef37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:41:06.379024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:06.380144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:06.381016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:06.381016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:06.381016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:06.381016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:06.381016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:41:06.381016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:41:06.381016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:41:06.381016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:41:06.381016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:41:06.381016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:41:06.381016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:06.381016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:41:06.381016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 574/1141] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:41:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:06.385982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8d693-195c-47f6-8824-8efedb1e4d7d None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:06.491240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac8d693-195c-47f6-8824-8efedb1e4d7d None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:06.525411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbb9a5-6702-40df-9780-80215d551259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:06.526034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbb9a5-6702-40df-9780-80215d551259 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:06.526317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbb9a5-6702-40df-9780-80215d551259 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:06.526589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dbb9a5-6702-40df-9780-80215d551259 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:06.527251 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbb9a5-6702-40df-9780-80215d551259 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:06.527552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbb9a5-6702-40df-9780-80215d551259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:06.528911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dbb9a5-6702-40df-9780-80215d551259 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:06.536709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 568/1142] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:41:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:06.542607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dadeb-c4c9-4bf9-b73b-c269d5bbef37 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:06.600063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:06.639034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772dadeb-c4c9-4bf9-b73b-c269d5bbef37 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:06.671317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d4bbc-1367-4fb2-9109-22f9931a9dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:06.671919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d4bbc-1367-4fb2-9109-22f9931a9dfa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:06.672192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d4bbc-1367-4fb2-9109-22f9931a9dfa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:06.672460 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d4bbc-1367-4fb2-9109-22f9931a9dfa None admin] PATH_INFO: `/v3/users/a9219c2e38584245be7bef3444a0ee79` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:06.685637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d4bbc-1367-4fb2-9109-22f9931a9dfa None admin] RBAC: Authorizing `identity:get_user(user_id=a9219c2e38584245be7bef3444a0ee79)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:06.687087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d4bbc-1367-4fb2-9109-22f9931a9dfa None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:06.688674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 575/1143] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:41:06 2022] GET /identity/v3/users/a9219c2e38584245be7bef3444a0ee79 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:06.695239 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbb9a5-6702-40df-9780-80215d551259 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:06.736002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:06.793154 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dbb9a5-6702-40df-9780-80215d551259 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:06.825978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d0d69-164e-458e-a1bb-e105cfb473b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:06.826611 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d0d69-164e-458e-a1bb-e105cfb473b0 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:06.826890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d0d69-164e-458e-a1bb-e105cfb473b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:06.827242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d0d69-164e-458e-a1bb-e105cfb473b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:06.892252 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d0d69-164e-458e-a1bb-e105cfb473b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:06.893678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d0d69-164e-458e-a1bb-e105cfb473b0 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:06.932551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:06.988832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 569/1144] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:41:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:41:06.995340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d4bbc-1367-4fb2-9109-22f9931a9dfa None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:07.090817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d4bbc-1367-4fb2-9109-22f9931a9dfa None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:07.107770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:07.124799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fea48b-0872-4346-8020-6be1109185d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:07.125416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fea48b-0872-4346-8020-6be1109185d1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:07.125692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fea48b-0872-4346-8020-6be1109185d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:07.125993 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fea48b-0872-4346-8020-6be1109185d1 None admin] PATH_INFO: `/v3/projects/0c1503b78d904f858c344ad9f18b6b7f` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:07.127084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fea48b-0872-4346-8020-6be1109185d1 None admin] RBAC: Authorizing `identity:get_project(project_id=0c1503b78d904f858c344ad9f18b6b7f)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:07.128497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fea48b-0872-4346-8020-6be1109185d1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:07.130157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 576/1145] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:41:06 2022] GET /identity/v3/projects/0c1503b78d904f858c344ad9f18b6b7f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:07.136961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d0d69-164e-458e-a1bb-e105cfb473b0 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:07.237748 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d0d69-164e-458e-a1bb-e105cfb473b0 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:07.270590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2163f-0dd6-4c48-8b8b-4cb97a8a446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:07.271183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2163f-0dd6-4c48-8b8b-4cb97a8a446e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:07.271461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2163f-0dd6-4c48-8b8b-4cb97a8a446e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:07.271731 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2163f-0dd6-4c48-8b8b-4cb97a8a446e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:07.272450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2163f-0dd6-4c48-8b8b-4cb97a8a446e None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=0c1503b78d904f858c344ad9f18b6b7f, user.id=a9219c2e38584245be7bef3444a0ee79) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:07.272757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2163f-0dd6-4c48-8b8b-4cb97a8a446e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:07.274201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2163f-0dd6-4c48-8b8b-4cb97a8a446e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:07.279383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 570/1146] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:41:07 2022] GET /identity/v3/role_assignments?user.id=a9219c2e38584245be7bef3444a0ee79&scope.project.id=0c1503b78d904f858c344ad9f18b6b7f&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:09.391380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f8bff-8d3a-44f5-8ee2-84a132cc561b None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:09.391886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f8bff-8d3a-44f5-8ee2-84a132cc561b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:09.391979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f8bff-8d3a-44f5-8ee2-84a132cc561b None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:09.393236 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 577/1147] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:09.398784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1f8f3-2cc7-44fc-a002-cb8b9fee62f0 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:09.399199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1f8f3-2cc7-44fc-a002-cb8b9fee62f0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:09.399323 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1f8f3-2cc7-44fc-a002-cb8b9fee62f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:09.406479 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:09.418155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-7ca1f8f3-2cc7-44fc-a002-cb8b9fee62f0 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:09.419334 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7ca1f8f3-2cc7-44fc-a002-cb8b9fee62f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:09.484386 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 571/1148] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:09.751486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f8bff-8d3a-44f5-8ee2-84a132cc561b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:09.849381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f8bff-8d3a-44f5-8ee2-84a132cc561b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:09.882483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebc578-2757-4e86-b221-44922845293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:09.883077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebc578-2757-4e86-b221-44922845293b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:09.883355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebc578-2757-4e86-b221-44922845293b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:09.883628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ebc578-2757-4e86-b221-44922845293b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:09.893329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ebc578-2757-4e86-b221-44922845293b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-d8ebc578-2757-4e86-b221-44922845293b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:41:09.898141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:09.899082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:09.900019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:09.900019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:09.900019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:09.900019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:09.900019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:41:09.900019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:41:09.900019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:41:09.900019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:41:09.900019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:41:09.900019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:41:09.900019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:09.900019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:41:09.900019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 578/1149] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:41:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:09.905203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1f8f3-2cc7-44fc-a002-cb8b9fee62f0 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:10.000233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1f8f3-2cc7-44fc-a002-cb8b9fee62f0 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:10.032848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c36da-40bf-463a-9aa8-b4b5fff518b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:10.033384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c36da-40bf-463a-9aa8-b4b5fff518b4 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:10.033659 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c36da-40bf-463a-9aa8-b4b5fff518b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:10.033954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c36da-40bf-463a-9aa8-b4b5fff518b4 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:10.034611 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c36da-40bf-463a-9aa8-b4b5fff518b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:10.034911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c36da-40bf-463a-9aa8-b4b5fff518b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:10.036262 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c36da-40bf-463a-9aa8-b4b5fff518b4 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:10.037567 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:10.045152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 572/1150] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:41:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:10.050845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebc578-2757-4e86-b221-44922845293b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:10.097153 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:10.148323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ebc578-2757-4e86-b221-44922845293b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:10.180442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174dfa0c-50ff-4f8c-9b24-5db7ac4f3fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:10.181048 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dfa0c-50ff-4f8c-9b24-5db7ac4f3fed None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:10.181324 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dfa0c-50ff-4f8c-9b24-5db7ac4f3fed None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:10.181593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dfa0c-50ff-4f8c-9b24-5db7ac4f3fed None admin] PATH_INFO: `/v3/users/a9219c2e38584245be7bef3444a0ee79` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:10.182713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174dfa0c-50ff-4f8c-9b24-5db7ac4f3fed None admin] RBAC: Authorizing `identity:get_user(user_id=a9219c2e38584245be7bef3444a0ee79)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:10.184124 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174dfa0c-50ff-4f8c-9b24-5db7ac4f3fed None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:10.185648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 579/1151] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:41:10 2022] GET /identity/v3/users/a9219c2e38584245be7bef3444a0ee79 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:10.191305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c36da-40bf-463a-9aa8-b4b5fff518b4 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:10.287528 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c36da-40bf-463a-9aa8-b4b5fff518b4 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:10.319068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c465d9-a711-4387-a286-9bcea52f91b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:10.320197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c465d9-a711-4387-a286-9bcea52f91b3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:10.320472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c465d9-a711-4387-a286-9bcea52f91b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:10.320741 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c465d9-a711-4387-a286-9bcea52f91b3 None admin] PATH_INFO: `/v3/projects/0c1503b78d904f858c344ad9f18b6b7f` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:10.321865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c465d9-a711-4387-a286-9bcea52f91b3 None admin] RBAC: Authorizing `identity:get_project(project_id=0c1503b78d904f858c344ad9f18b6b7f)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:10.323267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c465d9-a711-4387-a286-9bcea52f91b3 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:10.324842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 573/1152] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:41:10 2022] GET /identity/v3/projects/0c1503b78d904f858c344ad9f18b6b7f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:10.330500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174dfa0c-50ff-4f8c-9b24-5db7ac4f3fed None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:10.426009 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174dfa0c-50ff-4f8c-9b24-5db7ac4f3fed None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:10.439590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:10.460917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584cc9f-8be1-4adf-8ac7-8643e4c149a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:10.461570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584cc9f-8be1-4adf-8ac7-8643e4c149a6 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:10.461875 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584cc9f-8be1-4adf-8ac7-8643e4c149a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:10.462154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584cc9f-8be1-4adf-8ac7-8643e4c149a6 None admin] PATH_INFO: `/v3/projects/0c1503b78d904f858c344ad9f18b6b7f/users/a9219c2e38584245be7bef3444a0ee79/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:10.464037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584cc9f-8be1-4adf-8ac7-8643e4c149a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c1503b78d904f858c344ad9f18b6b7f, user_id=a9219c2e38584245be7bef3444a0ee79, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:10.465465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584cc9f-8be1-4adf-8ac7-8643e4c149a6 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:10.478213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 580/1153] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:41:10 2022] PUT /identity/v3/projects/0c1503b78d904f858c344ad9f18b6b7f/users/a9219c2e38584245be7bef3444a0ee79/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:41:12.574537 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38f9d0-0ed9-4b6c-88bb-e540ec52b224 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:12.575133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38f9d0-0ed9-4b6c-88bb-e540ec52b224 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:12.575133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38f9d0-0ed9-4b6c-88bb-e540ec52b224 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:12.576430 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 574/1154] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:12.581520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15968fce-862d-4353-a520-4e2546a781cf None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:12.581980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15968fce-862d-4353-a520-4e2546a781cf None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:12.582078 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15968fce-862d-4353-a520-4e2546a781cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:12.600359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-15968fce-862d-4353-a520-4e2546a781cf None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:12.601538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-15968fce-862d-4353-a520-4e2546a781cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:12.665836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 581/1155] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:12.939784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38f9d0-0ed9-4b6c-88bb-e540ec52b224 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:13.001986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:13.039666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38f9d0-0ed9-4b6c-88bb-e540ec52b224 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:13.072968 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad596db3-27d7-49f5-9241-08804414f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:13.073573 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad596db3-27d7-49f5-9241-08804414f1f4 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:13.073882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad596db3-27d7-49f5-9241-08804414f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:13.074155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad596db3-27d7-49f5-9241-08804414f1f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:13.083992 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad596db3-27d7-49f5-9241-08804414f1f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-ad596db3-27d7-49f5-9241-08804414f1f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:41:13.088824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:13.090035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:13.091163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:13.091163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:13.091163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:13.091163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:13.091163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:41:13.091163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:41:13.091163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:41:13.091163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:41:13.091163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:41:13.091163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:41:13.091163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:13.091163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:41:13.091163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 575/1156] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:41:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:13.095929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15968fce-862d-4353-a520-4e2546a781cf None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:13.191055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15968fce-862d-4353-a520-4e2546a781cf None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:13.193547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:13.225365 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187549d3-adac-4707-b103-89c373cb22b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:13.226458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187549d3-adac-4707-b103-89c373cb22b3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:13.226732 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187549d3-adac-4707-b103-89c373cb22b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:13.226996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187549d3-adac-4707-b103-89c373cb22b3 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:13.227645 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187549d3-adac-4707-b103-89c373cb22b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:13.227945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187549d3-adac-4707-b103-89c373cb22b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:13.229287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187549d3-adac-4707-b103-89c373cb22b3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:13.236967 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 582/1157] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:41:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:13.243375 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad596db3-27d7-49f5-9241-08804414f1f4 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:13.344339 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad596db3-27d7-49f5-9241-08804414f1f4 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:13.367719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:13.378686 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a4c72-c440-4cad-a486-87a855639612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:13.379302 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a4c72-c440-4cad-a486-87a855639612 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:13.379576 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a4c72-c440-4cad-a486-87a855639612 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:13.379843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a4c72-c440-4cad-a486-87a855639612 None admin] PATH_INFO: `/v3/users/a9219c2e38584245be7bef3444a0ee79` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:13.380947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a4c72-c440-4cad-a486-87a855639612 None admin] RBAC: Authorizing `identity:get_user(user_id=a9219c2e38584245be7bef3444a0ee79)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:13.382397 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a4c72-c440-4cad-a486-87a855639612 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:13.383954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 576/1158] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:41:13 2022] GET /identity/v3/users/a9219c2e38584245be7bef3444a0ee79 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:13.389791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187549d3-adac-4707-b103-89c373cb22b3 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:13.485800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187549d3-adac-4707-b103-89c373cb22b3 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:13.517408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e8b5f-0a36-4478-9a65-f64ea4971119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:13.518029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e8b5f-0a36-4478-9a65-f64ea4971119 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:13.518304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e8b5f-0a36-4478-9a65-f64ea4971119 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:13.518570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e8b5f-0a36-4478-9a65-f64ea4971119 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:13.553211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:13.586977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e8b5f-0a36-4478-9a65-f64ea4971119 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:13.588432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e8b5f-0a36-4478-9a65-f64ea4971119 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:13.681941 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 583/1159] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:41:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:41:13.687999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a4c72-c440-4cad-a486-87a855639612 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:13.782215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a4c72-c440-4cad-a486-87a855639612 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:13.814707 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f6a32-8a39-4b04-99e3-ce68ebc69502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:13.815314 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f6a32-8a39-4b04-99e3-ce68ebc69502 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:13.815586 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f6a32-8a39-4b04-99e3-ce68ebc69502 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:13.815852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f6a32-8a39-4b04-99e3-ce68ebc69502 None admin] PATH_INFO: `/v3/projects/0c1503b78d904f858c344ad9f18b6b7f` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:13.816943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f6a32-8a39-4b04-99e3-ce68ebc69502 None admin] RBAC: Authorizing `identity:get_project(project_id=0c1503b78d904f858c344ad9f18b6b7f)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:13.818635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533f6a32-8a39-4b04-99e3-ce68ebc69502 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:13.820242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 577/1160] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:41:13 2022] GET /identity/v3/projects/0c1503b78d904f858c344ad9f18b6b7f => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:13.826415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e8b5f-0a36-4478-9a65-f64ea4971119 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:13.888816 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:13.925175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e8b5f-0a36-4478-9a65-f64ea4971119 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:13.957889 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d09fd-4766-4bd0-809d-bcae4f506baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:13.958488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d09fd-4766-4bd0-809d-bcae4f506baa None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:13.958767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d09fd-4766-4bd0-809d-bcae4f506baa None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:13.959037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d09fd-4766-4bd0-809d-bcae4f506baa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:13.959757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d09fd-4766-4bd0-809d-bcae4f506baa None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=0c1503b78d904f858c344ad9f18b6b7f, user.id=a9219c2e38584245be7bef3444a0ee79) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:13.960578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d09fd-4766-4bd0-809d-bcae4f506baa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:13.962025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d09fd-4766-4bd0-809d-bcae4f506baa None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:13.967548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 584/1161] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:41:13 2022] GET /identity/v3/role_assignments?user.id=a9219c2e38584245be7bef3444a0ee79&scope.project.id=0c1503b78d904f858c344ad9f18b6b7f&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:16.092596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b71ba4-2954-453a-a640-fbe20dfdab75 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:16.093141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b71ba4-2954-453a-a640-fbe20dfdab75 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:16.093141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b71ba4-2954-453a-a640-fbe20dfdab75 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:16.094506 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 578/1162] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:16.099709 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f4385-2350-4765-a1aa-2ca4eca3c649 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:16.100084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f4385-2350-4765-a1aa-2ca4eca3c649 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:16.100252 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f4385-2350-4765-a1aa-2ca4eca3c649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:16.118340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-cc4f4385-2350-4765-a1aa-2ca4eca3c649 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:16.119490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cc4f4385-2350-4765-a1aa-2ca4eca3c649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:16.182359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 585/1163] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:16.444292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b71ba4-2954-453a-a640-fbe20dfdab75 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:16.503185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:16.544115 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b71ba4-2954-453a-a640-fbe20dfdab75 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:16.576884 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bb411-913f-417d-a8bd-2ef652daf541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:16.577475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bb411-913f-417d-a8bd-2ef652daf541 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:16.577770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bb411-913f-417d-a8bd-2ef652daf541 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:16.578045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bb411-913f-417d-a8bd-2ef652daf541 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:16.642657 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bb411-913f-417d-a8bd-2ef652daf541 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:16.644086 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bb411-913f-417d-a8bd-2ef652daf541 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:16.699529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:16.738763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 579/1164] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:41:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:41:16.745185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f4385-2350-4765-a1aa-2ca4eca3c649 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:16.801280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:16.842269 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f4385-2350-4765-a1aa-2ca4eca3c649 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:16.875558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38866a8b-7304-4145-bc49-5dbe0cb04ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:16.876185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38866a8b-7304-4145-bc49-5dbe0cb04ae2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:16.876464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38866a8b-7304-4145-bc49-5dbe0cb04ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:16.876734 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38866a8b-7304-4145-bc49-5dbe0cb04ae2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:16.883906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38866a8b-7304-4145-bc49-5dbe0cb04ae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:16.885336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38866a8b-7304-4145-bc49-5dbe0cb04ae2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-38866a8b-7304-4145-bc49-5dbe0cb04ae2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:16.896515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:16.897807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:41:16.898695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 586/1165] 10.222.0.55 () {56 vars in 1117 bytes} [Thu Sep 8 13:41:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:16.903694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bb411-913f-417d-a8bd-2ef652daf541 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:17.000261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bb411-913f-417d-a8bd-2ef652daf541 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:17.032281 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0528d7b-5f54-4efd-912f-fbb97f5e0166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:17.032876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0528d7b-5f54-4efd-912f-fbb97f5e0166 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:17.033153 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0528d7b-5f54-4efd-912f-fbb97f5e0166 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:17.033422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0528d7b-5f54-4efd-912f-fbb97f5e0166 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:17.034088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0528d7b-5f54-4efd-912f-fbb97f5e0166 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:17.034391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0528d7b-5f54-4efd-912f-fbb97f5e0166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:17.035737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0528d7b-5f54-4efd-912f-fbb97f5e0166 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:17.043141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 580/1166] 10.222.0.55 () {56 vars in 1115 bytes} [Thu Sep 8 13:41:16 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:19.131000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f59540-d241-4257-9d16-70e0fde2bb6d None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:19.131544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f59540-d241-4257-9d16-70e0fde2bb6d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:19.131544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f59540-d241-4257-9d16-70e0fde2bb6d None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:19.132887 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 587/1167] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:19.138195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac8cb1-0e7b-4e97-bb07-9a0db53308b4 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:19.138620 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac8cb1-0e7b-4e97-bb07-9a0db53308b4 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:19.138750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac8cb1-0e7b-4e97-bb07-9a0db53308b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:19.157008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-7fac8cb1-0e7b-4e97-bb07-9a0db53308b4 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:19.158265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7fac8cb1-0e7b-4e97-bb07-9a0db53308b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:19.175362 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:19.224461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 581/1168] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:19.494005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f59540-d241-4257-9d16-70e0fde2bb6d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:19.592104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f59540-d241-4257-9d16-70e0fde2bb6d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:19.611363 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:19.625062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17def786-4f18-41c6-be63-11e245a99cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:19.625654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17def786-4f18-41c6-be63-11e245a99cbc None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:19.625961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17def786-4f18-41c6-be63-11e245a99cbc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:19.626232 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17def786-4f18-41c6-be63-11e245a99cbc None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:19.626907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17def786-4f18-41c6-be63-11e245a99cbc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:19.628262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17def786-4f18-41c6-be63-11e245a99cbc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:19.642924 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 588/1169] 10.222.0.55 () {60 vars in 1138 bytes} [Thu Sep 8 13:41:19 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:41:21.767794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b1063-b977-4a84-a73a-28de861a27b5 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:21.768322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b1063-b977-4a84-a73a-28de861a27b5 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:21.768322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b1063-b977-4a84-a73a-28de861a27b5 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:21.769657 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 582/1170] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:21.775512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90fc3a-f63d-431c-8dba-d805f079914e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:21.775946 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90fc3a-f63d-431c-8dba-d805f079914e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:21.776056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa90fc3a-f63d-431c-8dba-d805f079914e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:21.793925 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-fa90fc3a-f63d-431c-8dba-d805f079914e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:21.795105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fa90fc3a-f63d-431c-8dba-d805f079914e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:21.859163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 589/1171] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:22.136860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b1063-b977-4a84-a73a-28de861a27b5 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:22.243189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b1063-b977-4a84-a73a-28de861a27b5 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:22.275624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d57da-091b-4602-9126-fead4508ac0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:22.276227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d57da-091b-4602-9126-fead4508ac0b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:22.276501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d57da-091b-4602-9126-fead4508ac0b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:22.276767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d57da-091b-4602-9126-fead4508ac0b None admin] PATH_INFO: `/v3/domains/7d8083e23d044fbbbdd8577b6e56dbeb` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:22.286893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d57da-091b-4602-9126-fead4508ac0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=7d8083e23d044fbbbdd8577b6e56dbeb)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:22.288818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d57da-091b-4602-9126-fead4508ac0b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:22.290399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 583/1172] 10.222.0.55 () {56 vars in 1183 bytes} [Thu Sep 8 13:41:22 2022] GET /identity/v3/domains/7d8083e23d044fbbbdd8577b6e56dbeb => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:22.296301 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90fc3a-f63d-431c-8dba-d805f079914e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:22.392818 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa90fc3a-f63d-431c-8dba-d805f079914e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:22.403242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:22.427898 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1844a10-34a6-4124-abf2-6de0013e5623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:22.428574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1844a10-34a6-4124-abf2-6de0013e5623 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:22.428856 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1844a10-34a6-4124-abf2-6de0013e5623 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:22.429136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1844a10-34a6-4124-abf2-6de0013e5623 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:22.429810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1844a10-34a6-4124-abf2-6de0013e5623 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:22.431239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1844a10-34a6-4124-abf2-6de0013e5623 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:22.458237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 590/1173] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:41:22 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:41:24.586039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15f5b0-c2b4-4150-b583-dc72e1c3b9f5 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:24.586591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15f5b0-c2b4-4150-b583-dc72e1c3b9f5 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:24.586591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15f5b0-c2b4-4150-b583-dc72e1c3b9f5 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:24.587916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 584/1174] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:24.593004 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd144990-47bc-46b3-ad0b-c55c2ed72942 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:24.593395 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd144990-47bc-46b3-ad0b-c55c2ed72942 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:24.593543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd144990-47bc-46b3-ad0b-c55c2ed72942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:24.611708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-cd144990-47bc-46b3-ad0b-c55c2ed72942 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:24.612863 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cd144990-47bc-46b3-ad0b-c55c2ed72942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:24.676940 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 591/1175] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:24.942545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15f5b0-c2b4-4150-b583-dc72e1c3b9f5 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:24.958591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:25.042545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15f5b0-c2b4-4150-b583-dc72e1c3b9f5 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:25.080137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a76f38-5816-4f42-bc73-ba5ffb03bd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:25.080751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a76f38-5816-4f42-bc73-ba5ffb03bd72 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:25.081024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a76f38-5816-4f42-bc73-ba5ffb03bd72 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:25.081292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a76f38-5816-4f42-bc73-ba5ffb03bd72 None admin] PATH_INFO: `/v3/domains/7d8083e23d044fbbbdd8577b6e56dbeb` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:25.082408 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a76f38-5816-4f42-bc73-ba5ffb03bd72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7d8083e23d044fbbbdd8577b6e56dbeb)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:25.083819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a76f38-5816-4f42-bc73-ba5ffb03bd72 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:25.085966 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 585/1176] 10.222.0.55 () {56 vars in 1183 bytes} [Thu Sep 8 13:41:24 2022] GET /identity/v3/domains/7d8083e23d044fbbbdd8577b6e56dbeb => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:25.091788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd144990-47bc-46b3-ad0b-c55c2ed72942 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:25.178845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:25.187891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd144990-47bc-46b3-ad0b-c55c2ed72942 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:25.221024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee412c-6f33-4089-849d-d0c72ee29417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:25.221622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee412c-6f33-4089-849d-d0c72ee29417 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:25.221932 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee412c-6f33-4089-849d-d0c72ee29417 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:25.222209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee412c-6f33-4089-849d-d0c72ee29417 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:25.222858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee412c-6f33-4089-849d-d0c72ee29417 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:25.224279 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee412c-6f33-4089-849d-d0c72ee29417 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:25.245643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 592/1177] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:41:25 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:41:27.402032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fb90a-42cd-4f00-8aff-67b46302fed3 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:27.402593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fb90a-42cd-4f00-8aff-67b46302fed3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:27.402593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fb90a-42cd-4f00-8aff-67b46302fed3 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:27.405708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 586/1178] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:41:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:27.411127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e89f4-1eac-4f49-8613-7fa54e1103dc None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:27.411524 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e89f4-1eac-4f49-8613-7fa54e1103dc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:27.411679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e89f4-1eac-4f49-8613-7fa54e1103dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:27.429906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9a5e89f4-1eac-4f49-8613-7fa54e1103dc None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:27.431080 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9a5e89f4-1eac-4f49-8613-7fa54e1103dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:27.496387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 593/1179] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:27.768658 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fb90a-42cd-4f00-8aff-67b46302fed3 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:27.848373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:27.867604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fb90a-42cd-4f00-8aff-67b46302fed3 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:27.900022 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb7c0a-2b25-4368-b1da-23d96b158e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:27.900643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb7c0a-2b25-4368-b1da-23d96b158e5e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:27.900920 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb7c0a-2b25-4368-b1da-23d96b158e5e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:27.901191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb7c0a-2b25-4368-b1da-23d96b158e5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:27.911264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb7c0a-2b25-4368-b1da-23d96b158e5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-cedb7c0a-2b25-4368-b1da-23d96b158e5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:41:27.916610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:27.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:27.918813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:27.918813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:27.918813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:27.918813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:27.918813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:41:27.918813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:41:27.918813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:41:27.918813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:41:27.918813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:41:27.918813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:41:27.918813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:27.918813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:41:27.918813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 587/1180] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:41:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:27.923633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e89f4-1eac-4f49-8613-7fa54e1103dc None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:27.998304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:28.021035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e89f4-1eac-4f49-8613-7fa54e1103dc None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:28.059663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864725cb-8ce2-49d4-9aba-9a86a65b90f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:28.060259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864725cb-8ce2-49d4-9aba-9a86a65b90f4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:28.060532 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864725cb-8ce2-49d4-9aba-9a86a65b90f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:28.060800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864725cb-8ce2-49d4-9aba-9a86a65b90f4 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:28.061458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864725cb-8ce2-49d4-9aba-9a86a65b90f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:28.061779 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864725cb-8ce2-49d4-9aba-9a86a65b90f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:28.063135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864725cb-8ce2-49d4-9aba-9a86a65b90f4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:28.071143 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 594/1181] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:41:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:28.077070 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb7c0a-2b25-4368-b1da-23d96b158e5e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:28.173827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb7c0a-2b25-4368-b1da-23d96b158e5e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:28.206000 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbaf875-6c0b-4190-8235-52857f63c3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:28.206609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbaf875-6c0b-4190-8235-52857f63c3f8 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:28.206886 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbaf875-6c0b-4190-8235-52857f63c3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:28.207160 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbaf875-6c0b-4190-8235-52857f63c3f8 None admin] PATH_INFO: `/v3/users/203d5a80658041d7a89f9d7e68638c97` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:28.218265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:28.220704 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbaf875-6c0b-4190-8235-52857f63c3f8 None admin] RBAC: Authorizing `identity:get_user(user_id=203d5a80658041d7a89f9d7e68638c97)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:28.222643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbaf875-6c0b-4190-8235-52857f63c3f8 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:28.224420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 588/1182] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:41:28 2022] GET /identity/v3/users/203d5a80658041d7a89f9d7e68638c97 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:28.230503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864725cb-8ce2-49d4-9aba-9a86a65b90f4 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:28.324669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864725cb-8ce2-49d4-9aba-9a86a65b90f4 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:28.356348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016e8ad-bb52-4251-b8c6-d8e4c5c5c081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:28.356932 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016e8ad-bb52-4251-b8c6-d8e4c5c5c081 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:28.357204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016e8ad-bb52-4251-b8c6-d8e4c5c5c081 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:28.357471 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016e8ad-bb52-4251-b8c6-d8e4c5c5c081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:28.367170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:28.423402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016e8ad-bb52-4251-b8c6-d8e4c5c5c081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:28.424823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016e8ad-bb52-4251-b8c6-d8e4c5c5c081 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:28.517496 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 595/1183] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:41:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:41:28.523448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbaf875-6c0b-4190-8235-52857f63c3f8 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:28.620896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbaf875-6c0b-4190-8235-52857f63c3f8 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:28.653411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e8ac3-f301-4c39-a3b8-481bcca1eff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:28.654059 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e8ac3-f301-4c39-a3b8-481bcca1eff4 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:28.654340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e8ac3-f301-4c39-a3b8-481bcca1eff4 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:28.654615 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e8ac3-f301-4c39-a3b8-481bcca1eff4 None admin] PATH_INFO: `/v3/projects/0884e83becf94d239e420d3cf7314afc` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:28.655723 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e8ac3-f301-4c39-a3b8-481bcca1eff4 None admin] RBAC: Authorizing `identity:get_project(project_id=0884e83becf94d239e420d3cf7314afc)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:28.657142 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411e8ac3-f301-4c39-a3b8-481bcca1eff4 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:28.658753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 589/1184] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:41:28 2022] GET /identity/v3/projects/0884e83becf94d239e420d3cf7314afc => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:28.664777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016e8ad-bb52-4251-b8c6-d8e4c5c5c081 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:28.704286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:28.760372 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016e8ad-bb52-4251-b8c6-d8e4c5c5c081 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:28.793665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2b1de-4cb0-4676-ae6c-c56fdd4a0a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:28.794327 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2b1de-4cb0-4676-ae6c-c56fdd4a0a2c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:28.794611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2b1de-4cb0-4676-ae6c-c56fdd4a0a2c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:28.794909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2b1de-4cb0-4676-ae6c-c56fdd4a0a2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:28.795628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2b1de-4cb0-4676-ae6c-c56fdd4a0a2c None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=0884e83becf94d239e420d3cf7314afc, user.id=203d5a80658041d7a89f9d7e68638c97) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:28.795937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2b1de-4cb0-4676-ae6c-c56fdd4a0a2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:28.797356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2b1de-4cb0-4676-ae6c-c56fdd4a0a2c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:28.802585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 596/1185] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:41:28 2022] GET /identity/v3/role_assignments?user.id=203d5a80658041d7a89f9d7e68638c97&scope.project.id=0884e83becf94d239e420d3cf7314afc&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:30.908234 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fd942-81ab-4764-9940-cbb912681ce0 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:30.908612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fd942-81ab-4764-9940-cbb912681ce0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:30.908612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fd942-81ab-4764-9940-cbb912681ce0 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:30.909976 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 590/1186] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:30.915753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e116e9-451d-4d08-9b87-3e4c8319962e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:30.916198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e116e9-451d-4d08-9b87-3e4c8319962e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:30.916293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e116e9-451d-4d08-9b87-3e4c8319962e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:30.934376 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-69e116e9-451d-4d08-9b87-3e4c8319962e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:30.935556 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-69e116e9-451d-4d08-9b87-3e4c8319962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:30.999830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 597/1187] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:31.271298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fd942-81ab-4764-9940-cbb912681ce0 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:31.337174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:31.370412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fd942-81ab-4764-9940-cbb912681ce0 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:31.403947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427b4d4-d1e9-48aa-ab49-9cdc2f1d8e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:31.404611 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427b4d4-d1e9-48aa-ab49-9cdc2f1d8e75 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:31.404902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427b4d4-d1e9-48aa-ab49-9cdc2f1d8e75 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:31.405175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427b4d4-d1e9-48aa-ab49-9cdc2f1d8e75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:31.415215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427b4d4-d1e9-48aa-ab49-9cdc2f1d8e75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-e427b4d4-d1e9-48aa-ab49-9cdc2f1d8e75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:41:31.419978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:31.421088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:31.422310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:31.422310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:31.422310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:31.422310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:31.422310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:41:31.422310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:41:31.422310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:41:31.422310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:41:31.422310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:41:31.422310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:41:31.422310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:31.422310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:41:31.422310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 591/1188] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:41:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:31.427024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e116e9-451d-4d08-9b87-3e4c8319962e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:31.464561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:31.528133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e116e9-451d-4d08-9b87-3e4c8319962e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:31.560695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de292a-e128-40c5-9ffc-212421bcd549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:31.561292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de292a-e128-40c5-9ffc-212421bcd549 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:31.561570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de292a-e128-40c5-9ffc-212421bcd549 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:31.561867 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de292a-e128-40c5-9ffc-212421bcd549 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:31.562521 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de292a-e128-40c5-9ffc-212421bcd549 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:31.562819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de292a-e128-40c5-9ffc-212421bcd549 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:31.564167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de292a-e128-40c5-9ffc-212421bcd549 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:31.572623 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 598/1189] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:41:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:31.578411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427b4d4-d1e9-48aa-ab49-9cdc2f1d8e75 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:31.673823 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427b4d4-d1e9-48aa-ab49-9cdc2f1d8e75 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:31.700081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:31.707921 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e30c9-0fbc-4e71-9198-5a709a8afc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:31.708532 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e30c9-0fbc-4e71-9198-5a709a8afc83 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:31.708808 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e30c9-0fbc-4e71-9198-5a709a8afc83 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:31.709103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e30c9-0fbc-4e71-9198-5a709a8afc83 None admin] PATH_INFO: `/v3/users/203d5a80658041d7a89f9d7e68638c97` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:31.710238 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e30c9-0fbc-4e71-9198-5a709a8afc83 None admin] RBAC: Authorizing `identity:get_user(user_id=203d5a80658041d7a89f9d7e68638c97)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:31.711656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e30c9-0fbc-4e71-9198-5a709a8afc83 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:31.713207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 592/1190] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:41:31 2022] GET /identity/v3/users/203d5a80658041d7a89f9d7e68638c97 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:31.718925 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de292a-e128-40c5-9ffc-212421bcd549 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:31.809708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:31.815658 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de292a-e128-40c5-9ffc-212421bcd549 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:31.848221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc4aad-ef69-4eb0-80fe-92e3862588f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:31.848834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc4aad-ef69-4eb0-80fe-92e3862588f1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:31.849112 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc4aad-ef69-4eb0-80fe-92e3862588f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:31.849384 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc4aad-ef69-4eb0-80fe-92e3862588f1 None admin] PATH_INFO: `/v3/projects/0884e83becf94d239e420d3cf7314afc` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:31.850522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc4aad-ef69-4eb0-80fe-92e3862588f1 None admin] RBAC: Authorizing `identity:get_project(project_id=0884e83becf94d239e420d3cf7314afc)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:31.851951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc4aad-ef69-4eb0-80fe-92e3862588f1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:31.853553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 599/1191] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:41:31 2022] GET /identity/v3/projects/0884e83becf94d239e420d3cf7314afc => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:31.859396 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e30c9-0fbc-4e71-9198-5a709a8afc83 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:31.955895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e30c9-0fbc-4e71-9198-5a709a8afc83 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:31.988224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29baf9aa-7e15-4198-b2c5-3c1e6afaa118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:31.988863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baf9aa-7e15-4198-b2c5-3c1e6afaa118 None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:31.989143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baf9aa-7e15-4198-b2c5-3c1e6afaa118 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:31.989413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29baf9aa-7e15-4198-b2c5-3c1e6afaa118 None admin] PATH_INFO: `/v3/projects/0884e83becf94d239e420d3cf7314afc/users/203d5a80658041d7a89f9d7e68638c97/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:31.991990 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baf9aa-7e15-4198-b2c5-3c1e6afaa118 None admin] RBAC: Authorizing `identity:create_grant(project_id=0884e83becf94d239e420d3cf7314afc, user_id=203d5a80658041d7a89f9d7e68638c97, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:31.993396 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29baf9aa-7e15-4198-b2c5-3c1e6afaa118 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:32.005321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 593/1192] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:41:31 2022] PUT /identity/v3/projects/0884e83becf94d239e420d3cf7314afc/users/203d5a80658041d7a89f9d7e68638c97/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:41:34.114822 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bc31e-5464-485d-95fe-38a72966f216 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:34.115836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bc31e-5464-485d-95fe-38a72966f216 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:34.115836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bc31e-5464-485d-95fe-38a72966f216 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:34.117169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 600/1193] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:41:34.122438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f9214-da95-4a44-ae33-5a4c74d40a6d None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:34.122874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f9214-da95-4a44-ae33-5a4c74d40a6d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:34.122992 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f9214-da95-4a44-ae33-5a4c74d40a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:34.141283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-883f9214-da95-4a44-ae33-5a4c74d40a6d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:41:34.142488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-883f9214-da95-4a44-ae33-5a4c74d40a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:34.191061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:34.208560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 594/1194] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:41:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:34.480083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bc31e-5464-485d-95fe-38a72966f216 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:34.578596 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bc31e-5464-485d-95fe-38a72966f216 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:34.612137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c7a84-c7dc-413e-8ed5-9897aa2fbeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:34.612753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c7a84-c7dc-413e-8ed5-9897aa2fbeba None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:34.613032 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c7a84-c7dc-413e-8ed5-9897aa2fbeba None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:34.613306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c7a84-c7dc-413e-8ed5-9897aa2fbeba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:34.624151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c7a84-c7dc-413e-8ed5-9897aa2fbeba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-004c7a84-c7dc-413e-8ed5-9897aa2fbeba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:41:34.628961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:34.630103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:34.631045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:34.631045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:34.631045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:34.631045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:34.631045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:41:34.631045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:41:34.631045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:41:34.631045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:41:34.631045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:41:34.631045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:41:34.631045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:34.631045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:41:34.631045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 601/1195] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:41:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:34.635980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f9214-da95-4a44-ae33-5a4c74d40a6d None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:34.732587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f9214-da95-4a44-ae33-5a4c74d40a6d None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:34.766815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d831ab-f54a-4dd7-b1e8-467667e71709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:34.767409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d831ab-f54a-4dd7-b1e8-467667e71709 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:34.767688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d831ab-f54a-4dd7-b1e8-467667e71709 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:34.767995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d831ab-f54a-4dd7-b1e8-467667e71709 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:34.768655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d831ab-f54a-4dd7-b1e8-467667e71709 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:34.768958 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d831ab-f54a-4dd7-b1e8-467667e71709 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:34.770346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d831ab-f54a-4dd7-b1e8-467667e71709 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:34.778206 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 595/1196] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:41:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:34.784019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c7a84-c7dc-413e-8ed5-9897aa2fbeba None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:34.809134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:34.880311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c7a84-c7dc-413e-8ed5-9897aa2fbeba None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:34.912611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676ff45-ad9f-4700-8121-af42951419e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:34.913230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676ff45-ad9f-4700-8121-af42951419e8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:34.913504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676ff45-ad9f-4700-8121-af42951419e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:34.913801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676ff45-ad9f-4700-8121-af42951419e8 None admin] PATH_INFO: `/v3/users/203d5a80658041d7a89f9d7e68638c97` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:34.914902 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676ff45-ad9f-4700-8121-af42951419e8 None admin] RBAC: Authorizing `identity:get_user(user_id=203d5a80658041d7a89f9d7e68638c97)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:34.916317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676ff45-ad9f-4700-8121-af42951419e8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:34.917878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 602/1197] 10.222.0.55 () {56 vars in 1177 bytes} [Thu Sep 8 13:41:34 2022] GET /identity/v3/users/203d5a80658041d7a89f9d7e68638c97 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:34.923609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d831ab-f54a-4dd7-b1e8-467667e71709 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:34.972091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:35.021363 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d831ab-f54a-4dd7-b1e8-467667e71709 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:35.053979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec186bf-7ad7-4f6b-915f-fa919d61e2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:35.054571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec186bf-7ad7-4f6b-915f-fa919d61e2cb None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:35.054852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec186bf-7ad7-4f6b-915f-fa919d61e2cb None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:35.055124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec186bf-7ad7-4f6b-915f-fa919d61e2cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:35.119345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec186bf-7ad7-4f6b-915f-fa919d61e2cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:35.120785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec186bf-7ad7-4f6b-915f-fa919d61e2cb None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:35.166718 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:35.217109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 596/1198] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:41:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:41:35.223062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676ff45-ad9f-4700-8121-af42951419e8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:35.309140 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:35.317928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676ff45-ad9f-4700-8121-af42951419e8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:35.350029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c289b-6fb7-4fad-aa20-b6ea095022fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:35.350632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c289b-6fb7-4fad-aa20-b6ea095022fb None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:35.350905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c289b-6fb7-4fad-aa20-b6ea095022fb None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:35.351171 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c289b-6fb7-4fad-aa20-b6ea095022fb None admin] PATH_INFO: `/v3/projects/0884e83becf94d239e420d3cf7314afc` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:35.352263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c289b-6fb7-4fad-aa20-b6ea095022fb None admin] RBAC: Authorizing `identity:get_project(project_id=0884e83becf94d239e420d3cf7314afc)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:35.353663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c289b-6fb7-4fad-aa20-b6ea095022fb None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:35.355267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 603/1199] 10.222.0.55 () {56 vars in 1186 bytes} [Thu Sep 8 13:41:35 2022] GET /identity/v3/projects/0884e83becf94d239e420d3cf7314afc => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:35.361194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec186bf-7ad7-4f6b-915f-fa919d61e2cb None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:35.457360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec186bf-7ad7-4f6b-915f-fa919d61e2cb None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:35.489371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab50cc-5a9b-488d-bc90-80e08b0c8db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:35.490648 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab50cc-5a9b-488d-bc90-80e08b0c8db8 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:35.490946 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab50cc-5a9b-488d-bc90-80e08b0c8db8 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:35.491254 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab50cc-5a9b-488d-bc90-80e08b0c8db8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:35.492577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab50cc-5a9b-488d-bc90-80e08b0c8db8 None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=0884e83becf94d239e420d3cf7314afc, user.id=203d5a80658041d7a89f9d7e68638c97) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:35.493114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab50cc-5a9b-488d-bc90-80e08b0c8db8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:35.494963 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab50cc-5a9b-488d-bc90-80e08b0c8db8 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:35.501675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 597/1200] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:41:35 2022] GET /identity/v3/role_assignments?user.id=203d5a80658041d7a89f9d7e68638c97&scope.project.id=0884e83becf94d239e420d3cf7314afc&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:45.346165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=20 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:45.499054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=94 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:31.082438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b6318-bc95-4d41-a008-436f5f795be3 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:31.083062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b6318-bc95-4d41-a008-436f5f795be3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:31.083062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b6318-bc95-4d41-a008-436f5f795be3 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:31.084497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 604/1201] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:42:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:31.090496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacfbae-c178-4754-abf5-828c4ded6367 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:31.090974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacfbae-c178-4754-abf5-828c4ded6367 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:31.091130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacfbae-c178-4754-abf5-828c4ded6367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:31.111987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-bbacfbae-c178-4754-abf5-828c4ded6367 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:31.113256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-bbacfbae-c178-4754-abf5-828c4ded6367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:31.208120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 598/1202] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:31.515591 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b6318-bc95-4d41-a008-436f5f795be3 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:31.642726 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b6318-bc95-4d41-a008-436f5f795be3 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:31.681172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671215b9-89a8-47ad-82d3-b7d2073fc9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:31.681831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671215b9-89a8-47ad-82d3-b7d2073fc9bd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:31.682116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671215b9-89a8-47ad-82d3-b7d2073fc9bd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:31.682391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671215b9-89a8-47ad-82d3-b7d2073fc9bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:31.695162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671215b9-89a8-47ad-82d3-b7d2073fc9bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:31.696626 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671215b9-89a8-47ad-82d3-b7d2073fc9bd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-671215b9-89a8-47ad-82d3-b7d2073fc9bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:31.713317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:31.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:31.722482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:31.722482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:31.722482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:31.722482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 605/1203] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:31.723962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacfbae-c178-4754-abf5-828c4ded6367 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:31.848034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacfbae-c178-4754-abf5-828c4ded6367 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:31.882304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:31.890863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573d762-2818-4a9b-b24a-e85906fdba90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:31.891551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573d762-2818-4a9b-b24a-e85906fdba90 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:31.891833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573d762-2818-4a9b-b24a-e85906fdba90 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:31.892105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573d762-2818-4a9b-b24a-e85906fdba90 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:31.892850 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573d762-2818-4a9b-b24a-e85906fdba90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:31.893153 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573d762-2818-4a9b-b24a-e85906fdba90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:31.895511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d573d762-2818-4a9b-b24a-e85906fdba90 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:31.915864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 599/1204] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:31.921349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671215b9-89a8-47ad-82d3-b7d2073fc9bd None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:31.987823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:32.053679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671215b9-89a8-47ad-82d3-b7d2073fc9bd None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:32.102941 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439310ac-2054-4d96-8588-a1e2c485aa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:32.103622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439310ac-2054-4d96-8588-a1e2c485aa81 None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:32.103901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439310ac-2054-4d96-8588-a1e2c485aa81 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:32.104200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439310ac-2054-4d96-8588-a1e2c485aa81 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:32.105299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439310ac-2054-4d96-8588-a1e2c485aa81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:32.107381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439310ac-2054-4d96-8588-a1e2c485aa81 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:32.140718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 606/1205] 10.222.0.55 () {60 vars in 1132 bytes} [Thu Sep 8 13:42:31 2022] POST /identity/v3/users => generated 251 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:42:34.564475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5aea63-e1d1-461e-9ba6-c5b459bbe881 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:34.565044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5aea63-e1d1-461e-9ba6-c5b459bbe881 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:34.565044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5aea63-e1d1-461e-9ba6-c5b459bbe881 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:34.567340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 600/1206] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:42:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:34.572747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec664b-9310-4b54-a28d-afa6b7249680 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:34.574957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec664b-9310-4b54-a28d-afa6b7249680 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:34.574957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aec664b-9310-4b54-a28d-afa6b7249680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:34.593540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5aec664b-9310-4b54-a28d-afa6b7249680 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:34.594781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5aec664b-9310-4b54-a28d-afa6b7249680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:34.681946 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 607/1207] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:34.953004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5aea63-e1d1-461e-9ba6-c5b459bbe881 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:35.070270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5aea63-e1d1-461e-9ba6-c5b459bbe881 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:35.116421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bd7af-0d2d-4f6b-9dc6-beec8a2a5b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:35.117135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bd7af-0d2d-4f6b-9dc6-beec8a2a5b52 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:35.117401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bd7af-0d2d-4f6b-9dc6-beec8a2a5b52 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:35.117677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bd7af-0d2d-4f6b-9dc6-beec8a2a5b52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:35.129000 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7bd7af-0d2d-4f6b-9dc6-beec8a2a5b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-8b7bd7af-0d2d-4f6b-9dc6-beec8a2a5b52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:35.133800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:35.134792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:35.135673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:35.135673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:35.135673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:35.135673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:35.135673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:35.135673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:35.135673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:35.135673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:35.135673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:35.135673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:35.135673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:35.135673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:35.135673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 601/1208] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:42:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:35.141908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec664b-9310-4b54-a28d-afa6b7249680 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:35.186666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:35.259470 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aec664b-9310-4b54-a28d-afa6b7249680 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:35.295894 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b8e3c-268a-405d-b8a8-f58d357d31d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:35.296731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b8e3c-268a-405d-b8a8-f58d357d31d9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:35.297021 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b8e3c-268a-405d-b8a8-f58d357d31d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:35.297294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b8e3c-268a-405d-b8a8-f58d357d31d9 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:35.298022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b8e3c-268a-405d-b8a8-f58d357d31d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:35.298381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b8e3c-268a-405d-b8a8-f58d357d31d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:35.300413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b8e3c-268a-405d-b8a8-f58d357d31d9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:35.309869 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 608/1209] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:42:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:35.316333 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bd7af-0d2d-4f6b-9dc6-beec8a2a5b52 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:35.380191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:35.438703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bd7af-0d2d-4f6b-9dc6-beec8a2a5b52 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:35.488490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135d259-3574-4f65-968f-347c72db28dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:35.489156 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135d259-3574-4f65-968f-347c72db28dd None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:35.489424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135d259-3574-4f65-968f-347c72db28dd None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:35.489739 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135d259-3574-4f65-968f-347c72db28dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:35.501486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135d259-3574-4f65-968f-347c72db28dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:35.503235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135d259-3574-4f65-968f-347c72db28dd None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-8135d259-3574-4f65-968f-347c72db28dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:35.519478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:35.520745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:35.521881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:35.521881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:35.521881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:35.521881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 602/1210] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:35.527021 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b8e3c-268a-405d-b8a8-f58d357d31d9 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:35.637229 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:35.641899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b8e3c-268a-405d-b8a8-f58d357d31d9 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:35.682031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4dc5a-12e5-4141-8533-508542a53bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:35.683031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4dc5a-12e5-4141-8533-508542a53bd4 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:35.683511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4dc5a-12e5-4141-8533-508542a53bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:35.683938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b4dc5a-12e5-4141-8533-508542a53bd4 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:35.684784 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4dc5a-12e5-4141-8533-508542a53bd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:35.685255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4dc5a-12e5-4141-8533-508542a53bd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:35.686900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b4dc5a-12e5-4141-8533-508542a53bd4 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:35.701530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 609/1211] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:35.707437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135d259-3574-4f65-968f-347c72db28dd None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:35.825550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135d259-3574-4f65-968f-347c72db28dd None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:35.827319 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:35.867371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed35c9-0ba7-4627-9f64-4146bc19233c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:35.868000 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed35c9-0ba7-4627-9f64-4146bc19233c None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:35.868276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed35c9-0ba7-4627-9f64-4146bc19233c None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:35.868547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed35c9-0ba7-4627-9f64-4146bc19233c None admin] PATH_INFO: `/v3/users/placement` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:35.876930 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed35c9-0ba7-4627-9f64-4146bc19233c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:35.878470 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed35c9-0ba7-4627-9f64-4146bc19233c None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-6aed35c9-0ba7-4627-9f64-4146bc19233c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:35.891487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:35.892563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:35.893572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:35.893572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:35.893572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:35.893572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:35.893572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:35.893572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:35.893572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:35.893572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:35.893572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:35.893572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:35.900231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 603/1212] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:35.900399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4dc5a-12e5-4141-8533-508542a53bd4 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:36.011079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b4dc5a-12e5-4141-8533-508542a53bd4 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:36.050543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ea8a4-1d92-4739-866f-a9e8415f3142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:36.052008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ea8a4-1d92-4739-866f-a9e8415f3142 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:36.052008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ea8a4-1d92-4739-866f-a9e8415f3142 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:36.052008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ea8a4-1d92-4739-866f-a9e8415f3142 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:36.052735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ea8a4-1d92-4739-866f-a9e8415f3142 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:36.053104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ea8a4-1d92-4739-866f-a9e8415f3142 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:36.054705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ea8a4-1d92-4739-866f-a9e8415f3142 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:36.073564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 610/1213] 10.222.0.55 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:36.079376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed35c9-0ba7-4627-9f64-4146bc19233c None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:36.187827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed35c9-0ba7-4627-9f64-4146bc19233c None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:36.224155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1c61b-461c-400c-b99a-6d20dae11f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:36.224765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1c61b-461c-400c-b99a-6d20dae11f11 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:36.225044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1c61b-461c-400c-b99a-6d20dae11f11 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:36.225313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1c61b-461c-400c-b99a-6d20dae11f11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:36.255801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:36.304994 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1c61b-461c-400c-b99a-6d20dae11f11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:36.306572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1c61b-461c-400c-b99a-6d20dae11f11 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:36.428688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 604/1214] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:42:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:42:36.434256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ea8a4-1d92-4739-866f-a9e8415f3142 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:36.460633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:36.560608 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ea8a4-1d92-4739-866f-a9e8415f3142 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:36.604098 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4e718-60dc-4af4-9426-846031f887df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:36.604772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4e718-60dc-4af4-9426-846031f887df None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:36.605044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4e718-60dc-4af4-9426-846031f887df None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:36.605320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4e718-60dc-4af4-9426-846031f887df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:36.616939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4e718-60dc-4af4-9426-846031f887df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:36.617883 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4e718-60dc-4af4-9426-846031f887df None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-8bc4e718-60dc-4af4-9426-846031f887df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:36.634120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:36.635314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:36.636359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:36.636359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:36.636359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:36.636359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 611/1215] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:36.641040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1c61b-461c-400c-b99a-6d20dae11f11 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:36.716433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:36.756533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1c61b-461c-400c-b99a-6d20dae11f11 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:36.794098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0663400-d5b9-4d35-9258-a0aa3cbcb7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:36.794736 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0663400-d5b9-4d35-9258-a0aa3cbcb7d9 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:36.795057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0663400-d5b9-4d35-9258-a0aa3cbcb7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:36.795398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0663400-d5b9-4d35-9258-a0aa3cbcb7d9 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:36.796120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0663400-d5b9-4d35-9258-a0aa3cbcb7d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:36.796424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0663400-d5b9-4d35-9258-a0aa3cbcb7d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:36.797918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0663400-d5b9-4d35-9258-a0aa3cbcb7d9 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:36.812797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 605/1216] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:36.818871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4e718-60dc-4af4-9426-846031f887df None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:36.903763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:36.939364 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4e718-60dc-4af4-9426-846031f887df None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:36.978271 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d271a-ea45-4571-8e4a-b5f805523214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:36.979045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d271a-ea45-4571-8e4a-b5f805523214 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:36.979446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d271a-ea45-4571-8e4a-b5f805523214 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:36.979841 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5d271a-ea45-4571-8e4a-b5f805523214 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:36.987113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d271a-ea45-4571-8e4a-b5f805523214 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:36.988784 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5d271a-ea45-4571-8e4a-b5f805523214 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-be5d271a-ea45-4571-8e4a-b5f805523214 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:37.003796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:37.005008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:37.008024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:37.008024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:37.008024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:37.008024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:37.008024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:37.008024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:37.008024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:37.008024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:37.009972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 612/1217] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:37.011834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0663400-d5b9-4d35-9258-a0aa3cbcb7d9 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:37.127245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0663400-d5b9-4d35-9258-a0aa3cbcb7d9 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:37.155376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:37.168764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee44266-ae60-4e64-bb68-fe3c7cfbd4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:37.169378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44266-ae60-4e64-bb68-fe3c7cfbd4d5 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:37.169655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44266-ae60-4e64-bb68-fe3c7cfbd4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:37.169980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44266-ae60-4e64-bb68-fe3c7cfbd4d5 None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:37.170680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee44266-ae60-4e64-bb68-fe3c7cfbd4d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:37.171072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee44266-ae60-4e64-bb68-fe3c7cfbd4d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:37.172842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee44266-ae60-4e64-bb68-fe3c7cfbd4d5 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:37.186918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 606/1218] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:37.193562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d271a-ea45-4571-8e4a-b5f805523214 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:37.295412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5d271a-ea45-4571-8e4a-b5f805523214 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:37.324905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:37.332398 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e339b-1afa-41a4-aedb-8cffb73899bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:37.333069 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e339b-1afa-41a4-aedb-8cffb73899bd None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:37.333349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e339b-1afa-41a4-aedb-8cffb73899bd None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:37.333618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e339b-1afa-41a4-aedb-8cffb73899bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:37.334391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e339b-1afa-41a4-aedb-8cffb73899bd None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=e42fb05dbae64c758098c591d6ffa790) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:37.334694 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e339b-1afa-41a4-aedb-8cffb73899bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:37.336350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e339b-1afa-41a4-aedb-8cffb73899bd None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:37.342320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 613/1219] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:42:37 2022] GET /identity/v3/role_assignments?user.id=e42fb05dbae64c758098c591d6ffa790&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:39.665752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbb4ee-34ae-4a0e-8c57-d1c4202e9b8f None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:39.666320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbb4ee-34ae-4a0e-8c57-d1c4202e9b8f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:39.666320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbb4ee-34ae-4a0e-8c57-d1c4202e9b8f None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:39.667627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 607/1220] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:42:39.672895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f52928-0b9a-4880-8259-3113610bd431 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:39.673318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f52928-0b9a-4880-8259-3113610bd431 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:39.673490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f52928-0b9a-4880-8259-3113610bd431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:39.694971 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b0f52928-0b9a-4880-8259-3113610bd431 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:39.696242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b0f52928-0b9a-4880-8259-3113610bd431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:39.773550 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 614/1221] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:42:40.056017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbb4ee-34ae-4a0e-8c57-d1c4202e9b8f None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:40.165427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbb4ee-34ae-4a0e-8c57-d1c4202e9b8f None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:40.206553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcec0c1-b4d4-43bb-8672-140d0da74ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:40.207484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcec0c1-b4d4-43bb-8672-140d0da74ff3 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:40.207888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcec0c1-b4d4-43bb-8672-140d0da74ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:40.208286 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcec0c1-b4d4-43bb-8672-140d0da74ff3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:40.218576 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcec0c1-b4d4-43bb-8672-140d0da74ff3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-9dcec0c1-b4d4-43bb-8672-140d0da74ff3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:40.223995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:40.225108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:40.226029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:40.226029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:40.226029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:40.226029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:40.226029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:40.226029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:40.226029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:40.226029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:40.226029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:40.226029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:40.226029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:40.226029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:40.226029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 608/1222] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:42:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:40.231600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f52928-0b9a-4880-8259-3113610bd431 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:40.344744 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f52928-0b9a-4880-8259-3113610bd431 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:40.370650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:40.384428 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f0c14-5778-46b1-89fb-9810f049d64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:40.385037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f0c14-5778-46b1-89fb-9810f049d64e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:40.385306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f0c14-5778-46b1-89fb-9810f049d64e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:40.385578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f0c14-5778-46b1-89fb-9810f049d64e None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:40.386303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f0c14-5778-46b1-89fb-9810f049d64e None admin] RBAC: Adding query filter params (name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:40.386568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f0c14-5778-46b1-89fb-9810f049d64e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:40.387926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86f0c14-5778-46b1-89fb-9810f049d64e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:40.396159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 615/1223] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:42:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:40.401906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcec0c1-b4d4-43bb-8672-140d0da74ff3 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:40.452261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:40.524313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcec0c1-b4d4-43bb-8672-140d0da74ff3 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:40.563221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e1aa0-b2a2-4d4c-b6e6-e5b592ad1fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:40.563852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e1aa0-b2a2-4d4c-b6e6-e5b592ad1fba None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:40.564134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e1aa0-b2a2-4d4c-b6e6-e5b592ad1fba None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:40.564408 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e1aa0-b2a2-4d4c-b6e6-e5b592ad1fba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:40.577118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e1aa0-b2a2-4d4c-b6e6-e5b592ad1fba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:40.578720 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e1aa0-b2a2-4d4c-b6e6-e5b592ad1fba None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-c91e1aa0-b2a2-4d4c-b6e6-e5b592ad1fba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:40.594851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:40.595856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:40.597683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:40.597683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:40.597683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:40.597683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 609/1224] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:40.602528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f0c14-5778-46b1-89fb-9810f049d64e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:40.707864 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f0c14-5778-46b1-89fb-9810f049d64e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:40.743192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2524d5-817e-49f8-a2da-a2aab597747e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:40.743856 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2524d5-817e-49f8-a2da-a2aab597747e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:40.744172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2524d5-817e-49f8-a2da-a2aab597747e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:40.744573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2524d5-817e-49f8-a2da-a2aab597747e None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:40.745368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2524d5-817e-49f8-a2da-a2aab597747e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:40.745847 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2524d5-817e-49f8-a2da-a2aab597747e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:40.747532 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2524d5-817e-49f8-a2da-a2aab597747e None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:40.760856 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 616/1225] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:40.766897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e1aa0-b2a2-4d4c-b6e6-e5b592ad1fba None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:40.869302 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:40.876940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e1aa0-b2a2-4d4c-b6e6-e5b592ad1fba None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:40.916956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422bd14-bbe9-4372-94a1-acfe586ba66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:40.917673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422bd14-bbe9-4372-94a1-acfe586ba66d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:40.918051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422bd14-bbe9-4372-94a1-acfe586ba66d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:40.918374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422bd14-bbe9-4372-94a1-acfe586ba66d None admin] PATH_INFO: `/v3/users/placement` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:40.927061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422bd14-bbe9-4372-94a1-acfe586ba66d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:40.928726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422bd14-bbe9-4372-94a1-acfe586ba66d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-b422bd14-bbe9-4372-94a1-acfe586ba66d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:40.940077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:40.942021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:40.943001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:40.943001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:40.943001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:40.943001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:40.943001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:40.943001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:40.943001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:40.943001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:40.943001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:40.943001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:40.943001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 610/1226] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:40.948468 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2524d5-817e-49f8-a2da-a2aab597747e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:41.001261 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:41.061620 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2524d5-817e-49f8-a2da-a2aab597747e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:41.097744 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7659a-79b0-4613-8e21-b4f5b4d19c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:41.098365 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7659a-79b0-4613-8e21-b4f5b4d19c61 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:41.098645 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7659a-79b0-4613-8e21-b4f5b4d19c61 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:41.098919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7659a-79b0-4613-8e21-b4f5b4d19c61 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:41.099648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7659a-79b0-4613-8e21-b4f5b4d19c61 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:41.099957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7659a-79b0-4613-8e21-b4f5b4d19c61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:41.101384 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee7659a-79b0-4613-8e21-b4f5b4d19c61 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:41.119995 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 617/1227] 10.222.0.55 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:41.126572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422bd14-bbe9-4372-94a1-acfe586ba66d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:41.234790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422bd14-bbe9-4372-94a1-acfe586ba66d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:41.269653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d41378-c020-4f3d-8342-91ab327ebe82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:41.270306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d41378-c020-4f3d-8342-91ab327ebe82 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:41.270584 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d41378-c020-4f3d-8342-91ab327ebe82 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:41.270853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d41378-c020-4f3d-8342-91ab327ebe82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:41.284256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d41378-c020-4f3d-8342-91ab327ebe82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:41.286122 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d41378-c020-4f3d-8342-91ab327ebe82 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-79d41378-c020-4f3d-8342-91ab327ebe82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:41.302873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:41.304133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:41.305079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:41.305079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:41.305079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:41.305079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 611/1228] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:41.310897 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7659a-79b0-4613-8e21-b4f5b4d19c61 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:41.421565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7659a-79b0-4613-8e21-b4f5b4d19c61 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:41.424125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:41.467688 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be0224-14c3-4187-aa20-c72235b83007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:41.468644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be0224-14c3-4187-aa20-c72235b83007 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:41.469358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be0224-14c3-4187-aa20-c72235b83007 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:41.469358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be0224-14c3-4187-aa20-c72235b83007 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:41.470596 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be0224-14c3-4187-aa20-c72235b83007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:41.471034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be0224-14c3-4187-aa20-c72235b83007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:41.473518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be0224-14c3-4187-aa20-c72235b83007 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:41.489718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 618/1229] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:41.497548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d41378-c020-4f3d-8342-91ab327ebe82 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:41.508202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:41.605143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d41378-c020-4f3d-8342-91ab327ebe82 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:41.640594 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86ba53-7e76-4362-bd86-4dd5a6ceef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:41.641477 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86ba53-7e76-4362-bd86-4dd5a6ceef6e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:41.642114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86ba53-7e76-4362-bd86-4dd5a6ceef6e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:41.642435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86ba53-7e76-4362-bd86-4dd5a6ceef6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:41.650916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86ba53-7e76-4362-bd86-4dd5a6ceef6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:41.652349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86ba53-7e76-4362-bd86-4dd5a6ceef6e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-fe86ba53-7e76-4362-bd86-4dd5a6ceef6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:41.664960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:41.666175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:41.667149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:41.667149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:41.667149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:41.667149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:41.667149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:41.667149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:41.667149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:41.667149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:41.667149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 612/1230] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:41.672015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be0224-14c3-4187-aa20-c72235b83007 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:41.785284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be0224-14c3-4187-aa20-c72235b83007 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:41.822760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d016598-d9f0-4b92-8627-4a0b9699ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:41.823379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d016598-d9f0-4b92-8627-4a0b9699ec09 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:41.823680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d016598-d9f0-4b92-8627-4a0b9699ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:41.823927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d016598-d9f0-4b92-8627-4a0b9699ec09 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:41.824615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d016598-d9f0-4b92-8627-4a0b9699ec09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:41.824927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d016598-d9f0-4b92-8627-4a0b9699ec09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:41.826404 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d016598-d9f0-4b92-8627-4a0b9699ec09 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:41.839246 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 619/1231] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:41.845309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86ba53-7e76-4362-bd86-4dd5a6ceef6e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:41.918449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:41.956830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86ba53-7e76-4362-bd86-4dd5a6ceef6e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:41.993648 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50f05f-e6ec-423b-a120-6a2e7aaad4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:41.994363 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50f05f-e6ec-423b-a120-6a2e7aaad4fd None admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:41.994642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50f05f-e6ec-423b-a120-6a2e7aaad4fd None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:41.994917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50f05f-e6ec-423b-a120-6a2e7aaad4fd None admin] PATH_INFO: `/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/e42fb05dbae64c758098c591d6ffa790/roles/7377909b229e4449a4cbe1a22f8bf1f8` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:42.011490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50f05f-e6ec-423b-a120-6a2e7aaad4fd None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe9c4e7bc14e70b57bf834cfff365f, user_id=e42fb05dbae64c758098c591d6ffa790, role_id=7377909b229e4449a4cbe1a22f8bf1f8)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:42.012933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50f05f-e6ec-423b-a120-6a2e7aaad4fd None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:42.025453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 613/1232] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:42:41 2022] PUT /identity/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/e42fb05dbae64c758098c591d6ffa790/roles/7377909b229e4449a4cbe1a22f8bf1f8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:42:44.160382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a2c59-79fd-4c13-bd88-7e367da23c3e None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:44.160974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a2c59-79fd-4c13-bd88-7e367da23c3e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:44.160974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a2c59-79fd-4c13-bd88-7e367da23c3e None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:44.162476 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 620/1233] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:42:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:44.168985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf3b58-f1ad-4e29-8825-ee9e267c82bc None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:44.169445 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf3b58-f1ad-4e29-8825-ee9e267c82bc None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:44.169532 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf3b58-f1ad-4e29-8825-ee9e267c82bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:44.190345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-8cbf3b58-f1ad-4e29-8825-ee9e267c82bc None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:44.192024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8cbf3b58-f1ad-4e29-8825-ee9e267c82bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:44.263915 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 614/1234] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:44.578781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a2c59-79fd-4c13-bd88-7e367da23c3e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:44.608414 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:44.692198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a2c59-79fd-4c13-bd88-7e367da23c3e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:44.732999 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f67b85-03b5-4a90-a7c0-55abd9ced8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:44.733636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f67b85-03b5-4a90-a7c0-55abd9ced8b9 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:44.733942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f67b85-03b5-4a90-a7c0-55abd9ced8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:44.734214 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f67b85-03b5-4a90-a7c0-55abd9ced8b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:44.746798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f67b85-03b5-4a90-a7c0-55abd9ced8b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-96f67b85-03b5-4a90-a7c0-55abd9ced8b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:44.754837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:44.756360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:44.757488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:44.757488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:44.757488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:44.757488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:44.757488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:44.757488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:44.757488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:44.757488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:44.757488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:44.757488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:44.757488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:44.757488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:44.757488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 621/1235] 10.222.0.55 () {56 vars in 1102 bytes} [Thu Sep 8 13:42:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:44.766481 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf3b58-f1ad-4e29-8825-ee9e267c82bc None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:44.823043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:44.886208 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf3b58-f1ad-4e29-8825-ee9e267c82bc None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:44.922641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87eb8ab-a7ff-456d-a378-03d70f484060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:44.923254 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87eb8ab-a7ff-456d-a378-03d70f484060 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:44.923545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87eb8ab-a7ff-456d-a378-03d70f484060 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:44.923946 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87eb8ab-a7ff-456d-a378-03d70f484060 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:44.924771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87eb8ab-a7ff-456d-a378-03d70f484060 None admin] RBAC: Adding query filter params (name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:44.925237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87eb8ab-a7ff-456d-a378-03d70f484060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:44.926804 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87eb8ab-a7ff-456d-a378-03d70f484060 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:44.936930 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 615/1236] 10.222.0.55 () {56 vars in 1103 bytes} [Thu Sep 8 13:42:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:44.943813 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f67b85-03b5-4a90-a7c0-55abd9ced8b9 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:45.037217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:45.053840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f67b85-03b5-4a90-a7c0-55abd9ced8b9 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:45.098880 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe4af8-c661-4557-ac32-d376d2e290f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:45.099519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe4af8-c661-4557-ac32-d376d2e290f3 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:45.099801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe4af8-c661-4557-ac32-d376d2e290f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:45.100077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe4af8-c661-4557-ac32-d376d2e290f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:45.111188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe4af8-c661-4557-ac32-d376d2e290f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:45.113753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe4af8-c661-4557-ac32-d376d2e290f3 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-f5fe4af8-c661-4557-ac32-d376d2e290f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:45.133743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:45.134845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:45.135907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:45.135907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:45.135907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:45.135907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 622/1237] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:45.140861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87eb8ab-a7ff-456d-a378-03d70f484060 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:45.244398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87eb8ab-a7ff-456d-a378-03d70f484060 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:45.256098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:45.283700 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3aa7b-1706-4320-80b7-0a7effd04bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:45.284393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3aa7b-1706-4320-80b7-0a7effd04bd7 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:45.284660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3aa7b-1706-4320-80b7-0a7effd04bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:45.284956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3aa7b-1706-4320-80b7-0a7effd04bd7 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:45.285637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3aa7b-1706-4320-80b7-0a7effd04bd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:45.285967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3aa7b-1706-4320-80b7-0a7effd04bd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:45.287323 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b3aa7b-1706-4320-80b7-0a7effd04bd7 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:45.301119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 616/1238] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:45.306924 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe4af8-c661-4557-ac32-d376d2e290f3 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:45.415445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe4af8-c661-4557-ac32-d376d2e290f3 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:45.454948 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab9c98-bce4-4993-a634-97401cf8b7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:45.455594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab9c98-bce4-4993-a634-97401cf8b7d1 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:45.455888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab9c98-bce4-4993-a634-97401cf8b7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:45.456197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab9c98-bce4-4993-a634-97401cf8b7d1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:45.459108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:45.465351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab9c98-bce4-4993-a634-97401cf8b7d1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:45.466995 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab9c98-bce4-4993-a634-97401cf8b7d1 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-d7ab9c98-bce4-4993-a634-97401cf8b7d1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:45.479014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:45.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:45.481126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:45.481126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:45.481126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:45.481126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:45.481126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:45.481126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:45.481126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:45.481126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:45.481126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:45.481126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:45.481126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 623/1239] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:45.486175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3aa7b-1706-4320-80b7-0a7effd04bd7 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:45.608946 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b3aa7b-1706-4320-80b7-0a7effd04bd7 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:45.646272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6162266-7df0-4edf-aefc-7e176394ff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:45.646885 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6162266-7df0-4edf-aefc-7e176394ff8b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:45.647176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6162266-7df0-4edf-aefc-7e176394ff8b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:45.647433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6162266-7df0-4edf-aefc-7e176394ff8b None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:45.648161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6162266-7df0-4edf-aefc-7e176394ff8b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:45.648461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6162266-7df0-4edf-aefc-7e176394ff8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:45.650102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6162266-7df0-4edf-aefc-7e176394ff8b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:45.673818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 617/1240] 10.222.0.55 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:45.675634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab9c98-bce4-4993-a634-97401cf8b7d1 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:45.786154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab9c98-bce4-4993-a634-97401cf8b7d1 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:45.829156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df77812-7507-4f93-a1b9-d6e9e62b84ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:45.829798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df77812-7507-4f93-a1b9-d6e9e62b84ab None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:45.830073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df77812-7507-4f93-a1b9-d6e9e62b84ab None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:45.830363 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df77812-7507-4f93-a1b9-d6e9e62b84ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:45.900926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:45.910564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df77812-7507-4f93-a1b9-d6e9e62b84ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:45.912745 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df77812-7507-4f93-a1b9-d6e9e62b84ab None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:46.028684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 624/1241] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:42:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:42:46.035120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6162266-7df0-4edf-aefc-7e176394ff8b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:46.065561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:46.146393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6162266-7df0-4edf-aefc-7e176394ff8b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:46.186696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138f383-b8f2-4f09-9bb6-e5031d8bf545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:46.187310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138f383-b8f2-4f09-9bb6-e5031d8bf545 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:46.187637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138f383-b8f2-4f09-9bb6-e5031d8bf545 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:46.187923 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138f383-b8f2-4f09-9bb6-e5031d8bf545 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:46.200624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138f383-b8f2-4f09-9bb6-e5031d8bf545 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:46.202085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138f383-b8f2-4f09-9bb6-e5031d8bf545 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-b138f383-b8f2-4f09-9bb6-e5031d8bf545 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:46.217643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:46.219157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:46.220192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:46.220192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:46.220192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:46.220192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 618/1242] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:46.225465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df77812-7507-4f93-a1b9-d6e9e62b84ab None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:46.338873 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:46.345263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df77812-7507-4f93-a1b9-d6e9e62b84ab None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:46.384935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604519e3-ed3a-4575-9c02-46dbe1a1cdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:46.385637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604519e3-ed3a-4575-9c02-46dbe1a1cdd2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:46.386039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604519e3-ed3a-4575-9c02-46dbe1a1cdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:46.386384 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604519e3-ed3a-4575-9c02-46dbe1a1cdd2 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:46.387108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604519e3-ed3a-4575-9c02-46dbe1a1cdd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:46.387419 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604519e3-ed3a-4575-9c02-46dbe1a1cdd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:46.388818 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604519e3-ed3a-4575-9c02-46dbe1a1cdd2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:46.403265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 625/1243] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:46.409021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138f383-b8f2-4f09-9bb6-e5031d8bf545 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:46.497841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:46.526567 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138f383-b8f2-4f09-9bb6-e5031d8bf545 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:46.563339 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54534a2-ba96-4c3b-946e-d48c1da402e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:46.563964 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54534a2-ba96-4c3b-946e-d48c1da402e7 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:46.564244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54534a2-ba96-4c3b-946e-d48c1da402e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:46.564508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54534a2-ba96-4c3b-946e-d48c1da402e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:46.572853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54534a2-ba96-4c3b-946e-d48c1da402e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:46.574459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54534a2-ba96-4c3b-946e-d48c1da402e7 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-b54534a2-ba96-4c3b-946e-d48c1da402e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:46.588255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:46.589421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:46.590534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:46.590534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:46.590534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:46.590534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:46.590534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:46.590534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:46.590534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:46.590534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:46.590534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 619/1244] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:46.595852 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604519e3-ed3a-4575-9c02-46dbe1a1cdd2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:46.706751 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604519e3-ed3a-4575-9c02-46dbe1a1cdd2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:46.744748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216a136-b3a9-4a09-9989-b91ed6cbb678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:46.745352 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216a136-b3a9-4a09-9989-b91ed6cbb678 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:46.745632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216a136-b3a9-4a09-9989-b91ed6cbb678 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:46.745958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216a136-b3a9-4a09-9989-b91ed6cbb678 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:46.746651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216a136-b3a9-4a09-9989-b91ed6cbb678 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:46.746947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216a136-b3a9-4a09-9989-b91ed6cbb678 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:46.748378 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216a136-b3a9-4a09-9989-b91ed6cbb678 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:46.761294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 626/1245] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:46.767252 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54534a2-ba96-4c3b-946e-d48c1da402e7 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:46.881659 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54534a2-ba96-4c3b-946e-d48c1da402e7 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:46.920838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71cb6d-1cca-48e3-8d6c-2d2d2b452b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:46.921439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71cb6d-1cca-48e3-8d6c-2d2d2b452b37 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:46.921718 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71cb6d-1cca-48e3-8d6c-2d2d2b452b37 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:46.922276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71cb6d-1cca-48e3-8d6c-2d2d2b452b37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:46.923008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71cb6d-1cca-48e3-8d6c-2d2d2b452b37 None admin] RBAC: Adding query filter params (role.id=7377909b229e4449a4cbe1a22f8bf1f8, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=e42fb05dbae64c758098c591d6ffa790) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:46.923335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71cb6d-1cca-48e3-8d6c-2d2d2b452b37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:46.925252 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71cb6d-1cca-48e3-8d6c-2d2d2b452b37 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:46.931187 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 620/1246] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:42:46 2022] GET /identity/v3/role_assignments?user.id=e42fb05dbae64c758098c591d6ffa790&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=7377909b229e4449a4cbe1a22f8bf1f8 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:49.314184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc584e-e93e-49aa-b170-9e4663c13029 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:49.315264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc584e-e93e-49aa-b170-9e4663c13029 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:49.315264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc584e-e93e-49aa-b170-9e4663c13029 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:49.316961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 627/1247] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:42:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:49.322580 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1b13e-11df-4d66-9db5-be3b80e28e64 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:49.323107 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1b13e-11df-4d66-9db5-be3b80e28e64 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:49.323194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1b13e-11df-4d66-9db5-be3b80e28e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:49.329373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:49.343601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-89f1b13e-11df-4d66-9db5-be3b80e28e64 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:49.344937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-89f1b13e-11df-4d66-9db5-be3b80e28e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:49.417220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 621/1248] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:49.713288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc584e-e93e-49aa-b170-9e4663c13029 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:49.735858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:49.833465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc584e-e93e-49aa-b170-9e4663c13029 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:49.870520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7416e7-25a6-4f4b-b874-1237a18aa940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:49.871292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7416e7-25a6-4f4b-b874-1237a18aa940 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:49.871657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7416e7-25a6-4f4b-b874-1237a18aa940 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:49.871993 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7416e7-25a6-4f4b-b874-1237a18aa940 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:49.884421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7416e7-25a6-4f4b-b874-1237a18aa940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-0e7416e7-25a6-4f4b-b874-1237a18aa940 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:49.889310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:49.890523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:49.891662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:49.891662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:49.891662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:49.891662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:49.891662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:49.891662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:49.891662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:49.891662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:49.891662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:49.891662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:49.891662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:49.891662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:49.891662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 628/1249] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:42:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:49.896574 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1b13e-11df-4d66-9db5-be3b80e28e64 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:50.003408 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1b13e-11df-4d66-9db5-be3b80e28e64 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:50.038203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:50.044061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9daf5-793e-4632-90f0-b9c2f4e4a06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:50.044683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9daf5-793e-4632-90f0-b9c2f4e4a06d None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:50.044962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9daf5-793e-4632-90f0-b9c2f4e4a06d None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:50.045233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9daf5-793e-4632-90f0-b9c2f4e4a06d None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:50.045947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9daf5-793e-4632-90f0-b9c2f4e4a06d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:50.046251 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9daf5-793e-4632-90f0-b9c2f4e4a06d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:50.047639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9daf5-793e-4632-90f0-b9c2f4e4a06d None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:50.057195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 622/1250] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:42:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:50.063201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7416e7-25a6-4f4b-b874-1237a18aa940 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:50.143762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:50.167513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7416e7-25a6-4f4b-b874-1237a18aa940 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:50.208470 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a13106-3a66-48f6-8f55-8d07b84c3cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:50.209518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a13106-3a66-48f6-8f55-8d07b84c3cd7 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:50.209984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a13106-3a66-48f6-8f55-8d07b84c3cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:50.210483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a13106-3a66-48f6-8f55-8d07b84c3cd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:50.221955 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a13106-3a66-48f6-8f55-8d07b84c3cd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:50.223402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a13106-3a66-48f6-8f55-8d07b84c3cd7 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-a5a13106-3a66-48f6-8f55-8d07b84c3cd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:50.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:50.240684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:50.241817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:50.241817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:50.241817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:50.241817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 629/1251] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:50.247874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9daf5-793e-4632-90f0-b9c2f4e4a06d None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:50.350629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9daf5-793e-4632-90f0-b9c2f4e4a06d None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:50.385429 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d75671-848d-4ae0-bf47-72a4fa179c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:50.386062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75671-848d-4ae0-bf47-72a4fa179c4e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:50.386336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75671-848d-4ae0-bf47-72a4fa179c4e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:50.386600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d75671-848d-4ae0-bf47-72a4fa179c4e None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:50.387232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d75671-848d-4ae0-bf47-72a4fa179c4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:50.387531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d75671-848d-4ae0-bf47-72a4fa179c4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:50.388887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d75671-848d-4ae0-bf47-72a4fa179c4e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:50.401266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 623/1252] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:50.406891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a13106-3a66-48f6-8f55-8d07b84c3cd7 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:50.518717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a13106-3a66-48f6-8f55-8d07b84c3cd7 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:50.544023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:50.552081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda3d5d-060e-4950-b9ec-725a64177578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:50.552699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda3d5d-060e-4950-b9ec-725a64177578 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:50.552975 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda3d5d-060e-4950-b9ec-725a64177578 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:50.553242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda3d5d-060e-4950-b9ec-725a64177578 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:50.560381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda3d5d-060e-4950-b9ec-725a64177578 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:50.561831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda3d5d-060e-4950-b9ec-725a64177578 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-dcda3d5d-060e-4950-b9ec-725a64177578 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:50.572835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:50.574010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:50.574991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:50.574991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:50.574991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:50.574991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:50.574991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:50.574991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:50.574991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:50.574991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:50.574991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:50.574991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:50.574991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 630/1253] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:50.579748 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d75671-848d-4ae0-bf47-72a4fa179c4e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:50.633114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:50.675717 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d75671-848d-4ae0-bf47-72a4fa179c4e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:50.707965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9233d-e620-4f9d-8530-9bfd65936335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:50.708548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9233d-e620-4f9d-8530-9bfd65936335 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:50.708821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9233d-e620-4f9d-8530-9bfd65936335 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:50.709086 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9233d-e620-4f9d-8530-9bfd65936335 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:50.709822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9233d-e620-4f9d-8530-9bfd65936335 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:50.710125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9233d-e620-4f9d-8530-9bfd65936335 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:50.711526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9233d-e620-4f9d-8530-9bfd65936335 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:50.728149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 624/1254] 10.222.0.55 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:50.733772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda3d5d-060e-4950-b9ec-725a64177578 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:50.828912 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda3d5d-060e-4950-b9ec-725a64177578 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:50.860805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8502547-0b74-4d8f-ba70-771c6ec54811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:50.861391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8502547-0b74-4d8f-ba70-771c6ec54811 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:50.861668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8502547-0b74-4d8f-ba70-771c6ec54811 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:50.861966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8502547-0b74-4d8f-ba70-771c6ec54811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:50.914704 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:50.926968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8502547-0b74-4d8f-ba70-771c6ec54811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:50.928438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8502547-0b74-4d8f-ba70-771c6ec54811 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:51.022551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 631/1255] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:42:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:42:51.028335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9233d-e620-4f9d-8530-9bfd65936335 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:51.124814 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9233d-e620-4f9d-8530-9bfd65936335 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:51.138061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:51.158826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fc5b3-110c-41b2-ac98-748a46cb4c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:51.159378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fc5b3-110c-41b2-ac98-748a46cb4c86 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:51.159655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fc5b3-110c-41b2-ac98-748a46cb4c86 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:51.159925 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5fc5b3-110c-41b2-ac98-748a46cb4c86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:51.169689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fc5b3-110c-41b2-ac98-748a46cb4c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:51.171149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5fc5b3-110c-41b2-ac98-748a46cb4c86 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-1e5fc5b3-110c-41b2-ac98-748a46cb4c86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:51.185917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:51.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:51.187964 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:51.187964 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:51.187964 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:51.187964 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 625/1256] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:51.193685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8502547-0b74-4d8f-ba70-771c6ec54811 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:51.277377 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:51.289877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8502547-0b74-4d8f-ba70-771c6ec54811 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:51.323848 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24873ee6-8687-4c29-9433-a338c7897d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:51.324445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24873ee6-8687-4c29-9433-a338c7897d63 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:51.324724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24873ee6-8687-4c29-9433-a338c7897d63 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:51.324996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24873ee6-8687-4c29-9433-a338c7897d63 None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:51.325632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24873ee6-8687-4c29-9433-a338c7897d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:51.325967 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24873ee6-8687-4c29-9433-a338c7897d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:51.327324 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24873ee6-8687-4c29-9433-a338c7897d63 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:51.338105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 632/1257] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:51.343732 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fc5b3-110c-41b2-ac98-748a46cb4c86 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:51.439684 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5fc5b3-110c-41b2-ac98-748a46cb4c86 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:51.480209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6fc71-27b1-4efb-8fbd-a95e60fc3845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:51.480834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6fc71-27b1-4efb-8fbd-a95e60fc3845 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:51.481111 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6fc71-27b1-4efb-8fbd-a95e60fc3845 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:51.481394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6fc71-27b1-4efb-8fbd-a95e60fc3845 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:51.489081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6fc71-27b1-4efb-8fbd-a95e60fc3845 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:51.490559 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6fc71-27b1-4efb-8fbd-a95e60fc3845 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-eef6fc71-27b1-4efb-8fbd-a95e60fc3845 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:51.502467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:51.503531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:51.504667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:51.504667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:51.504667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:51.504667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:51.504667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:51.504667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:51.504667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:51.504667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:51.504667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 626/1258] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:51.511232 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24873ee6-8687-4c29-9433-a338c7897d63 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:51.610073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24873ee6-8687-4c29-9433-a338c7897d63 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:51.642155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234a924-6c62-4b77-b4c4-d29915a1a9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:51.642748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234a924-6c62-4b77-b4c4-d29915a1a9d8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:51.643023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234a924-6c62-4b77-b4c4-d29915a1a9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:51.643293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234a924-6c62-4b77-b4c4-d29915a1a9d8 None admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:51.643972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234a924-6c62-4b77-b4c4-d29915a1a9d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:51.644272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234a924-6c62-4b77-b4c4-d29915a1a9d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:51.645685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234a924-6c62-4b77-b4c4-d29915a1a9d8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:51.657149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 633/1259] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:51.663537 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6fc71-27b1-4efb-8fbd-a95e60fc3845 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:51.690047 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:51.761861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6fc71-27b1-4efb-8fbd-a95e60fc3845 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:51.794041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ff684-11fb-4de7-b3fd-893ac5e631e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:51.794627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ff684-11fb-4de7-b3fd-893ac5e631e1 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:51.794909 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ff684-11fb-4de7-b3fd-893ac5e631e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:51.795183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ff684-11fb-4de7-b3fd-893ac5e631e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:51.795896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ff684-11fb-4de7-b3fd-893ac5e631e1 None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=e42fb05dbae64c758098c591d6ffa790) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:51.796201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ff684-11fb-4de7-b3fd-893ac5e631e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:51.797606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ff684-11fb-4de7-b3fd-893ac5e631e1 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:51.803334 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 627/1260] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:42:51 2022] GET /identity/v3/role_assignments?user.id=e42fb05dbae64c758098c591d6ffa790&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:53.898301 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69437d-7d8a-4046-b9fb-cfd15c0451a8 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:53.898826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69437d-7d8a-4046-b9fb-cfd15c0451a8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:53.898826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69437d-7d8a-4046-b9fb-cfd15c0451a8 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:53.900177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 634/1261] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:53.905303 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefe582-f288-465b-8033-8b6c2c288618 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:53.905713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefe582-f288-465b-8033-8b6c2c288618 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:53.905891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefe582-f288-465b-8033-8b6c2c288618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:53.923681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-dcefe582-f288-465b-8033-8b6c2c288618 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:53.924878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-dcefe582-f288-465b-8033-8b6c2c288618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:53.989779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 628/1262] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:54.252492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69437d-7d8a-4046-b9fb-cfd15c0451a8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:54.263792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:54.351906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69437d-7d8a-4046-b9fb-cfd15c0451a8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:54.384807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63550b16-da96-4c03-a17c-6e8b8b1f232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:54.385415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63550b16-da96-4c03-a17c-6e8b8b1f232e None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:54.385691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63550b16-da96-4c03-a17c-6e8b8b1f232e None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:54.385987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63550b16-da96-4c03-a17c-6e8b8b1f232e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:54.395871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63550b16-da96-4c03-a17c-6e8b8b1f232e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-63550b16-da96-4c03-a17c-6e8b8b1f232e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:54.400614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:54.401701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:54.402783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:54.402783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:54.402783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:54.402783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:54.402783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:54.402783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:54.402783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:54.402783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:54.402783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:54.402783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:54.402783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:54.402783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:54.402783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 635/1263] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:42:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:54.407809 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefe582-f288-465b-8033-8b6c2c288618 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:54.425932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:54.518033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefe582-f288-465b-8033-8b6c2c288618 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:54.551632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc78b8-e703-4207-b7aa-5a8e706bf3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:54.552232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc78b8-e703-4207-b7aa-5a8e706bf3e7 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:54.552510 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc78b8-e703-4207-b7aa-5a8e706bf3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:54.552782 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc78b8-e703-4207-b7aa-5a8e706bf3e7 None admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:54.553443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc78b8-e703-4207-b7aa-5a8e706bf3e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:54.553766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc78b8-e703-4207-b7aa-5a8e706bf3e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:54.555123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc78b8-e703-4207-b7aa-5a8e706bf3e7 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:54.562878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 629/1264] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:42:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:54.568906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63550b16-da96-4c03-a17c-6e8b8b1f232e None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:54.633988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:54.666145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63550b16-da96-4c03-a17c-6e8b8b1f232e None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:54.698746 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2aa90c-5932-4058-9b2d-d2b19eee7905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:54.699362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2aa90c-5932-4058-9b2d-d2b19eee7905 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:54.699638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2aa90c-5932-4058-9b2d-d2b19eee7905 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:54.699906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2aa90c-5932-4058-9b2d-d2b19eee7905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:54.709539 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2aa90c-5932-4058-9b2d-d2b19eee7905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:54.710997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2aa90c-5932-4058-9b2d-d2b19eee7905 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-3a2aa90c-5932-4058-9b2d-d2b19eee7905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:54.725549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:54.726724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:54.727783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:54.727783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:54.727783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:54.727783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 636/1265] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:54.732337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc78b8-e703-4207-b7aa-5a8e706bf3e7 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:54.811436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:54.830696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc78b8-e703-4207-b7aa-5a8e706bf3e7 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:54.863531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c59d3-f181-4a1e-a353-848a5e5e8ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:54.864125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c59d3-f181-4a1e-a353-848a5e5e8ee8 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:54.864401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c59d3-f181-4a1e-a353-848a5e5e8ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:54.864671 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c59d3-f181-4a1e-a353-848a5e5e8ee8 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:54.865309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c59d3-f181-4a1e-a353-848a5e5e8ee8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:54.865610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c59d3-f181-4a1e-a353-848a5e5e8ee8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:54.866988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c59d3-f181-4a1e-a353-848a5e5e8ee8 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:54.878214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 630/1266] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:54.883808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2aa90c-5932-4058-9b2d-d2b19eee7905 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:54.989982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2aa90c-5932-4058-9b2d-d2b19eee7905 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:55.008545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:55.022863 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd881c81-0b66-4088-a6f8-98f0ec995909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:55.023485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd881c81-0b66-4088-a6f8-98f0ec995909 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:55.023766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd881c81-0b66-4088-a6f8-98f0ec995909 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:55.024040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd881c81-0b66-4088-a6f8-98f0ec995909 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:55.031186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd881c81-0b66-4088-a6f8-98f0ec995909 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:55.032623 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd881c81-0b66-4088-a6f8-98f0ec995909 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-bd881c81-0b66-4088-a6f8-98f0ec995909 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:55.044075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:55.045183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:55.046525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:55.046525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:55.046525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:55.046525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:55.046525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:55.046525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:55.046525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:55.046525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:55.046525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:55.046525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:55.046525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 637/1267] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:55.051045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c59d3-f181-4a1e-a353-848a5e5e8ee8 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:55.146521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c59d3-f181-4a1e-a353-848a5e5e8ee8 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:55.178987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc770e3-cbb2-46da-a3e1-170a6ed77e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:55.179578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc770e3-cbb2-46da-a3e1-170a6ed77e46 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:55.179856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc770e3-cbb2-46da-a3e1-170a6ed77e46 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:55.180128 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc770e3-cbb2-46da-a3e1-170a6ed77e46 None admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:55.180855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc770e3-cbb2-46da-a3e1-170a6ed77e46 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:55.181161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc770e3-cbb2-46da-a3e1-170a6ed77e46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:55.182598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc770e3-cbb2-46da-a3e1-170a6ed77e46 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:55.191247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:55.200154 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 631/1268] 10.222.0.55 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:55.206469 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd881c81-0b66-4088-a6f8-98f0ec995909 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:55.317625 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd881c81-0b66-4088-a6f8-98f0ec995909 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:55.350522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaee427-e201-4e39-afb0-4fe42236d49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:55.351142 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaee427-e201-4e39-afb0-4fe42236d49f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:55.351437 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaee427-e201-4e39-afb0-4fe42236d49f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:55.351692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaee427-e201-4e39-afb0-4fe42236d49f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:55.361997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaee427-e201-4e39-afb0-4fe42236d49f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:55.363426 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaee427-e201-4e39-afb0-4fe42236d49f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-acaee427-e201-4e39-afb0-4fe42236d49f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:55.377582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:55.378730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:55.379636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:55.379636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:55.379636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:55.379636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 638/1269] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:55.384524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc770e3-cbb2-46da-a3e1-170a6ed77e46 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:55.490229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc770e3-cbb2-46da-a3e1-170a6ed77e46 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:55.522490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec7fd4-450a-42a7-aa42-a4d009351a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:55.523089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec7fd4-450a-42a7-aa42-a4d009351a8a None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:55.523367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec7fd4-450a-42a7-aa42-a4d009351a8a None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:55.523638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec7fd4-450a-42a7-aa42-a4d009351a8a None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:55.524272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec7fd4-450a-42a7-aa42-a4d009351a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:55.524600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec7fd4-450a-42a7-aa42-a4d009351a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:55.525984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec7fd4-450a-42a7-aa42-a4d009351a8a None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:55.537334 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 632/1270] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:55.543066 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaee427-e201-4e39-afb0-4fe42236d49f None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:55.574238 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:55.652376 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaee427-e201-4e39-afb0-4fe42236d49f None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:55.684849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd2381-30fa-4f15-8555-a550bcef0a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:55.685475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd2381-30fa-4f15-8555-a550bcef0a67 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:55.685771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd2381-30fa-4f15-8555-a550bcef0a67 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:55.686048 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd2381-30fa-4f15-8555-a550bcef0a67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:55.692798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd2381-30fa-4f15-8555-a550bcef0a67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:55.694260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd2381-30fa-4f15-8555-a550bcef0a67 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-e4fd2381-30fa-4f15-8555-a550bcef0a67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:55.705755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:55.706770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:55.708013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:55.708013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:55.708013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:55.708013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:55.708013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:55.708013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:55.708013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:55.708013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:55.708013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 639/1271] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:55.712665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec7fd4-450a-42a7-aa42-a4d009351a8a None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:55.775546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:55.817761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec7fd4-450a-42a7-aa42-a4d009351a8a None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:55.855134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2212ad-7b6e-401e-8667-2c0f8d34fb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:55.855726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2212ad-7b6e-401e-8667-2c0f8d34fb60 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:55.856000 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2212ad-7b6e-401e-8667-2c0f8d34fb60 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:55.856263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2212ad-7b6e-401e-8667-2c0f8d34fb60 None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:55.856939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2212ad-7b6e-401e-8667-2c0f8d34fb60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:55.857236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2212ad-7b6e-401e-8667-2c0f8d34fb60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:55.859712 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2212ad-7b6e-401e-8667-2c0f8d34fb60 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:55.870903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 633/1272] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:55.878960 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd2381-30fa-4f15-8555-a550bcef0a67 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:55.973850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:55.983045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd2381-30fa-4f15-8555-a550bcef0a67 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:56.016640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb234c-98b6-43f4-9985-0ccac09fd9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:56.018055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb234c-98b6-43f4-9985-0ccac09fd9c9 None admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:56.018336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb234c-98b6-43f4-9985-0ccac09fd9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:56.018610 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb234c-98b6-43f4-9985-0ccac09fd9c9 None admin] PATH_INFO: `/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/e42fb05dbae64c758098c591d6ffa790/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:56.020486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb234c-98b6-43f4-9985-0ccac09fd9c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0bbe9c4e7bc14e70b57bf834cfff365f, user_id=e42fb05dbae64c758098c591d6ffa790, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:56.022459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb234c-98b6-43f4-9985-0ccac09fd9c9 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:56.034922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 640/1273] 10.222.0.55 () {58 vars in 1439 bytes} [Thu Sep 8 13:42:55 2022] PUT /identity/v3/projects/0bbe9c4e7bc14e70b57bf834cfff365f/users/e42fb05dbae64c758098c591d6ffa790/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:42:58.166403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c1b2c-a48e-45b2-a618-0d03deb86ad0 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:58.166917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c1b2c-a48e-45b2-a618-0d03deb86ad0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:58.166917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c1b2c-a48e-45b2-a618-0d03deb86ad0 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:58.168177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 634/1274] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:58.173354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac3eae-e47b-4172-8211-808554d7b73c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:58.173722 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac3eae-e47b-4172-8211-808554d7b73c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:58.173956 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac3eae-e47b-4172-8211-808554d7b73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:58.191871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c2ac3eae-e47b-4172-8211-808554d7b73c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:58.193041 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c2ac3eae-e47b-4172-8211-808554d7b73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:58.256787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 641/1275] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:58.541510 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c1b2c-a48e-45b2-a618-0d03deb86ad0 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:58.646465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c1b2c-a48e-45b2-a618-0d03deb86ad0 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:58.662997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:58.680196 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f8d56-b507-46d6-b5b7-617a22ecba14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:58.680810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f8d56-b507-46d6-b5b7-617a22ecba14 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:58.681090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f8d56-b507-46d6-b5b7-617a22ecba14 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:58.681362 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f8d56-b507-46d6-b5b7-617a22ecba14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:58.692012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f8d56-b507-46d6-b5b7-617a22ecba14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-416f8d56-b507-46d6-b5b7-617a22ecba14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:58.696806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:58.697979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:58.699046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:58.699046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:58.699046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:58.699046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:58.699046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:58.699046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:58.699046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:58.699046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:58.699046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:58.699046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:58.699046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:58.699046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:58.699046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 635/1276] 10.222.0.55 () {56 vars in 1096 bytes} [Thu Sep 8 13:42:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:58.704170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac3eae-e47b-4172-8211-808554d7b73c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:58.781930 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:58.801485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac3eae-e47b-4172-8211-808554d7b73c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:58.834952 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e3971-4c88-4c39-b17a-26a27e961849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:58.835547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e3971-4c88-4c39-b17a-26a27e961849 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:58.835822 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e3971-4c88-4c39-b17a-26a27e961849 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:58.836091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e3971-4c88-4c39-b17a-26a27e961849 None admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:58.836740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e3971-4c88-4c39-b17a-26a27e961849 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:58.837036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e3971-4c88-4c39-b17a-26a27e961849 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:58.838467 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e3971-4c88-4c39-b17a-26a27e961849 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:58.846173 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 642/1277] 10.222.0.55 () {56 vars in 1099 bytes} [Thu Sep 8 13:42:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:58.851883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f8d56-b507-46d6-b5b7-617a22ecba14 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:58.951780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f8d56-b507-46d6-b5b7-617a22ecba14 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:58.984252 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0db7c-3aba-4ca1-855b-5aa9a8ede42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:58.984850 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0db7c-3aba-4ca1-855b-5aa9a8ede42b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:58.985128 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0db7c-3aba-4ca1-855b-5aa9a8ede42b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:58.985398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0db7c-3aba-4ca1-855b-5aa9a8ede42b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:58.995614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0db7c-3aba-4ca1-855b-5aa9a8ede42b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:58.997051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0db7c-3aba-4ca1-855b-5aa9a8ede42b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-edf0db7c-3aba-4ca1-855b-5aa9a8ede42b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:59.011133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:59.012299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:59.013223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:59.013223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:59.013223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:59.013223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 636/1278] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:59.018237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e3971-4c88-4c39-b17a-26a27e961849 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:59.116223 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e3971-4c88-4c39-b17a-26a27e961849 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:59.147835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8b058-ece8-48d2-9b8a-b78a41ebcf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:59.148432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8b058-ece8-48d2-9b8a-b78a41ebcf7b None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:59.148712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8b058-ece8-48d2-9b8a-b78a41ebcf7b None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:59.148987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8b058-ece8-48d2-9b8a-b78a41ebcf7b None admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:59.149620 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8b058-ece8-48d2-9b8a-b78a41ebcf7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:59.149950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8b058-ece8-48d2-9b8a-b78a41ebcf7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:59.151300 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8b058-ece8-48d2-9b8a-b78a41ebcf7b None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:59.162988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 643/1279] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:59.168577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0db7c-3aba-4ca1-855b-5aa9a8ede42b None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:59.198938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:59.267734 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0db7c-3aba-4ca1-855b-5aa9a8ede42b None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:59.301151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bcfd5-6ced-4852-81cd-ba22a36f0281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:59.301800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bcfd5-6ced-4852-81cd-ba22a36f0281 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:59.302081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bcfd5-6ced-4852-81cd-ba22a36f0281 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:59.302350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bcfd5-6ced-4852-81cd-ba22a36f0281 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:59.309689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bcfd5-6ced-4852-81cd-ba22a36f0281 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:59.311142 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bcfd5-6ced-4852-81cd-ba22a36f0281 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-570bcfd5-6ced-4852-81cd-ba22a36f0281 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:59.322894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:59.323903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:59.325033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:59.325033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:59.325033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:59.325033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:59.325033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:59.325033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:59.325033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:59.325033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:59.325033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:59.325033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:42:59.325033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 637/1280] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:59.331108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8b058-ece8-48d2-9b8a-b78a41ebcf7b None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:59.334871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:59.429701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8b058-ece8-48d2-9b8a-b78a41ebcf7b None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:59.466366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb1258-a978-4aa8-99e9-681e2699e7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:59.466967 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb1258-a978-4aa8-99e9-681e2699e7a8 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:59.467253 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb1258-a978-4aa8-99e9-681e2699e7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:59.467528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb1258-a978-4aa8-99e9-681e2699e7a8 None admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:59.468247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb1258-a978-4aa8-99e9-681e2699e7a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:59.468551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb1258-a978-4aa8-99e9-681e2699e7a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:59.469992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb1258-a978-4aa8-99e9-681e2699e7a8 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:59.487236 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 644/1281] 10.222.0.55 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:59.493147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bcfd5-6ced-4852-81cd-ba22a36f0281 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:59.579673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:59.589678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bcfd5-6ced-4852-81cd-ba22a36f0281 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:59.621872 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d31e8b-f612-4d91-9af3-122ba78cb9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:59.622463 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d31e8b-f612-4d91-9af3-122ba78cb9cf None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:59.622736 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d31e8b-f612-4d91-9af3-122ba78cb9cf None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:59.623000 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d31e8b-f612-4d91-9af3-122ba78cb9cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:59.688327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d31e8b-f612-4d91-9af3-122ba78cb9cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:59.689684 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d31e8b-f612-4d91-9af3-122ba78cb9cf None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:59.774692 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:59.784644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 638/1282] 10.222.0.55 () {58 vars in 1282 bytes} [Thu Sep 8 13:42:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:42:59.790455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb1258-a978-4aa8-99e9-681e2699e7a8 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:59.849276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:59.886253 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb1258-a978-4aa8-99e9-681e2699e7a8 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:59.918918 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113213f-6cc5-483f-9307-f50c8678dab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:59.919537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113213f-6cc5-483f-9307-f50c8678dab2 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:59.919810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113213f-6cc5-483f-9307-f50c8678dab2 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:59.920076 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113213f-6cc5-483f-9307-f50c8678dab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:59.929665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113213f-6cc5-483f-9307-f50c8678dab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:59.931098 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113213f-6cc5-483f-9307-f50c8678dab2 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-1113213f-6cc5-483f-9307-f50c8678dab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:59.945877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:59.947002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:59.947935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:59.947935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:59.947935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:42:59.947935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 645/1283] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:59.952533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d31e8b-f612-4d91-9af3-122ba78cb9cf None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:00.046996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d31e8b-f612-4d91-9af3-122ba78cb9cf None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:00.079445 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b467c9-633b-425f-aa88-d65ccb3fabb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:00.080569 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b467c9-633b-425f-aa88-d65ccb3fabb6 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:00.080846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b467c9-633b-425f-aa88-d65ccb3fabb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:00.081114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b467c9-633b-425f-aa88-d65ccb3fabb6 None admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:00.081769 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b467c9-633b-425f-aa88-d65ccb3fabb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:43:00.082065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b467c9-633b-425f-aa88-d65ccb3fabb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:00.083407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b467c9-633b-425f-aa88-d65ccb3fabb6 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:00.094088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 639/1284] 10.222.0.55 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:43:00.099415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113213f-6cc5-483f-9307-f50c8678dab2 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:00.194947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113213f-6cc5-483f-9307-f50c8678dab2 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:00.207781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:00.227170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0e2ea-67b8-420e-89aa-18c17d10c504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:00.227782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0e2ea-67b8-420e-89aa-18c17d10c504 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:00.228058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0e2ea-67b8-420e-89aa-18c17d10c504 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:00.228327 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0e2ea-67b8-420e-89aa-18c17d10c504 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:00.234662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0e2ea-67b8-420e-89aa-18c17d10c504 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:00.236582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0e2ea-67b8-420e-89aa-18c17d10c504 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application [None req-8ea0e2ea-67b8-420e-89aa-18c17d10c504 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:43:00.247453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application with Lock( Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:43:00.248477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:43:00.254671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:43:00.254671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:43:00.254671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:43:00.254671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:43:00.254671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:43:00.254671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:43:00.254671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:43:00.254671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: ERROR keystone.server.flask.application Sep 08 13:43:00.254671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 646/1285] 10.222.0.55 () {56 vars in 1111 bytes} [Thu Sep 8 13:43:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:43:00.255287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b467c9-633b-425f-aa88-d65ccb3fabb6 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:00.304721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:00.352527 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b467c9-633b-425f-aa88-d65ccb3fabb6 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:00.384715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652d696-845e-4450-8b6a-5846b8ce6982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:00.385303 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652d696-845e-4450-8b6a-5846b8ce6982 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:00.385579 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652d696-845e-4450-8b6a-5846b8ce6982 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:00.385876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652d696-845e-4450-8b6a-5846b8ce6982 None admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:00.386557 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652d696-845e-4450-8b6a-5846b8ce6982 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:43:00.386857 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652d696-845e-4450-8b6a-5846b8ce6982 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:00.388261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652d696-845e-4450-8b6a-5846b8ce6982 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:00.399640 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 640/1286] 10.222.0.55 () {56 vars in 1148 bytes} [Thu Sep 8 13:43:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:43:00.405607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0e2ea-67b8-420e-89aa-18c17d10c504 None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:00.511041 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0e2ea-67b8-420e-89aa-18c17d10c504 None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:00.543126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a40343-a6ee-4eb1-9e83-c341a5c9ac95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:00.543720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a40343-a6ee-4eb1-9e83-c341a5c9ac95 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:00.543995 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a40343-a6ee-4eb1-9e83-c341a5c9ac95 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:00.544262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a40343-a6ee-4eb1-9e83-c341a5c9ac95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:00.544976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a40343-a6ee-4eb1-9e83-c341a5c9ac95 None admin] RBAC: Adding query filter params (role.id=398624259b8d4e2ea5beebc4f2ecbf57, scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f, user.id=e42fb05dbae64c758098c591d6ffa790) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:43:00.545282 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a40343-a6ee-4eb1-9e83-c341a5c9ac95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:00.546721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a40343-a6ee-4eb1-9e83-c341a5c9ac95 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:00.552312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 647/1287] 10.222.0.55 () {56 vars in 1374 bytes} [Thu Sep 8 13:43:00 2022] GET /identity/v3/role_assignments?user.id=e42fb05dbae64c758098c591d6ffa790&scope.project.id=0bbe9c4e7bc14e70b57bf834cfff365f&role.id=398624259b8d4e2ea5beebc4f2ecbf57 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:43:02.708927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9613760-3d41-4a89-a47b-355d599797a6 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:02.709479 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9613760-3d41-4a89-a47b-355d599797a6 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:02.709479 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9613760-3d41-4a89-a47b-355d599797a6 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:02.710839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 641/1288] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:43:02.715983 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29041a76-fceb-47bc-9ea7-c421db255b85 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:02.716403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29041a76-fceb-47bc-9ea7-c421db255b85 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:02.716538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29041a76-fceb-47bc-9ea7-c421db255b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:02.734961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-29041a76-fceb-47bc-9ea7-c421db255b85 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:43:02.736155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-29041a76-fceb-47bc-9ea7-c421db255b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:43:02.764846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:02.801501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 648/1289] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:43:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:43:03.067462 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9613760-3d41-4a89-a47b-355d599797a6 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:03.165665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9613760-3d41-4a89-a47b-355d599797a6 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:03.176428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:03.200010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b9ca9-c15f-433f-8fb6-88d5b8285189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:03.200629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b9ca9-c15f-433f-8fb6-88d5b8285189 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:03.200904 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b9ca9-c15f-433f-8fb6-88d5b8285189 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:03.201171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b9ca9-c15f-433f-8fb6-88d5b8285189 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:03.201759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b9ca9-c15f-433f-8fb6-88d5b8285189 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:03.203119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b9ca9-c15f-433f-8fb6-88d5b8285189 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-206b9ca9-c15f-433f-8fb6-88d5b8285189 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:43:03.211663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:43:03.212858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:43:03.213770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 642/1290] 10.222.0.55 () {56 vars in 1117 bytes} [Thu Sep 8 13:43:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:43:03.218634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29041a76-fceb-47bc-9ea7-c421db255b85 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:03.313268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29041a76-fceb-47bc-9ea7-c421db255b85 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:03.346329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180b111-c777-4e21-9c30-aa3ec73bc03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:03.346864 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180b111-c777-4e21-9c30-aa3ec73bc03f None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:03.347144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180b111-c777-4e21-9c30-aa3ec73bc03f None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:03.347415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180b111-c777-4e21-9c30-aa3ec73bc03f None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:03.348040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180b111-c777-4e21-9c30-aa3ec73bc03f None admin] RBAC: Adding query filter params (name=placement) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:43:03.348344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180b111-c777-4e21-9c30-aa3ec73bc03f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:03.349693 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180b111-c777-4e21-9c30-aa3ec73bc03f None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:03.354458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 649/1291] 10.222.0.55 () {56 vars in 1116 bytes} [Thu Sep 8 13:43:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:43:03.360004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b9ca9-c15f-433f-8fb6-88d5b8285189 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:03.461846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b9ca9-c15f-433f-8fb6-88d5b8285189 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:03.503253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ddee7-9334-4493-8111-12a417bc569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:03.503890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ddee7-9334-4493-8111-12a417bc569b None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:03.504163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ddee7-9334-4493-8111-12a417bc569b None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:03.504501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ddee7-9334-4493-8111-12a417bc569b None admin] PATH_INFO: `/v3/services` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:03.505168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ddee7-9334-4493-8111-12a417bc569b None admin] RBAC: Adding query filter params (type=placement) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:43:03.505546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ddee7-9334-4493-8111-12a417bc569b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:03.507055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ddee7-9334-4493-8111-12a417bc569b None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:03.511786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 643/1292] 10.222.0.55 () {56 vars in 1116 bytes} [Thu Sep 8 13:43:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:43:05.628481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24621a-79d2-4f48-b804-be64fcfe8286 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:05.629026 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24621a-79d2-4f48-b804-be64fcfe8286 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:05.629026 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24621a-79d2-4f48-b804-be64fcfe8286 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:05.630437 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 650/1293] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:43:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:43:05.635504 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9292042-f17e-4f57-829a-3162dc792729 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:05.635884 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9292042-f17e-4f57-829a-3162dc792729 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:05.636058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9292042-f17e-4f57-829a-3162dc792729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:05.657109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b9292042-f17e-4f57-829a-3162dc792729 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:43:05.658813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b9292042-f17e-4f57-829a-3162dc792729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:43:05.691630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:05.728129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 644/1294] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:43:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:43:05.997284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24621a-79d2-4f48-b804-be64fcfe8286 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:06.027919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:06.095862 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24621a-79d2-4f48-b804-be64fcfe8286 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:06.128015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cb54b-a449-4b2d-96a1-0b2289d5c7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:06.128604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cb54b-a449-4b2d-96a1-0b2289d5c7cc None admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:06.128877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cb54b-a449-4b2d-96a1-0b2289d5c7cc None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:06.129145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cb54b-a449-4b2d-96a1-0b2289d5c7cc None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:06.129815 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16cb54b-a449-4b2d-96a1-0b2289d5c7cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:06.131155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16cb54b-a449-4b2d-96a1-0b2289d5c7cc None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:06.145059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 651/1295] 10.222.0.55 () {60 vars in 1141 bytes} [Thu Sep 8 13:43:05 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:43:08.301015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba648899-0e93-4ff7-91e4-0a30ceabe30a None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:08.301538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba648899-0e93-4ff7-91e4-0a30ceabe30a None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:08.301538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba648899-0e93-4ff7-91e4-0a30ceabe30a None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:08.302865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 645/1296] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:43:08.308083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7da60a-4a88-4257-8be8-fc257db325cc None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:08.308447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7da60a-4a88-4257-8be8-fc257db325cc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:08.308635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7da60a-4a88-4257-8be8-fc257db325cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:08.330497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-3d7da60a-4a88-4257-8be8-fc257db325cc None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:43:08.332198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3d7da60a-4a88-4257-8be8-fc257db325cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:43:08.409432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 652/1297] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:43:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:43:08.683454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba648899-0e93-4ff7-91e4-0a30ceabe30a None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:08.781587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba648899-0e93-4ff7-91e4-0a30ceabe30a None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:08.814811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daf48f-e25c-408c-a800-b7d3e6ef1d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:08.815679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daf48f-e25c-408c-a800-b7d3e6ef1d31 None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:08.815965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daf48f-e25c-408c-a800-b7d3e6ef1d31 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:08.816237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daf48f-e25c-408c-a800-b7d3e6ef1d31 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:08.816813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daf48f-e25c-408c-a800-b7d3e6ef1d31 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:08.818201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daf48f-e25c-408c-a800-b7d3e6ef1d31 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-51daf48f-e25c-408c-a800-b7d3e6ef1d31 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:43:08.826699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:43:08.827776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:43:08.828617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 646/1298] 10.222.0.55 () {56 vars in 1117 bytes} [Thu Sep 8 13:43:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:43:08.833704 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7da60a-4a88-4257-8be8-fc257db325cc None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:08.851814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:08.930559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7da60a-4a88-4257-8be8-fc257db325cc None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:08.962760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe0904-a6aa-4e02-a20d-b3a570c1f60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:08.963304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe0904-a6aa-4e02-a20d-b3a570c1f60a None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:08.963582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe0904-a6aa-4e02-a20d-b3a570c1f60a None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:08.963853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe0904-a6aa-4e02-a20d-b3a570c1f60a None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:08.964474 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe0904-a6aa-4e02-a20d-b3a570c1f60a None admin] RBAC: Adding query filter params (name=placement) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:43:08.964781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe0904-a6aa-4e02-a20d-b3a570c1f60a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:08.966166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe0904-a6aa-4e02-a20d-b3a570c1f60a None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:08.971617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 653/1299] 10.222.0.55 () {56 vars in 1116 bytes} [Thu Sep 8 13:43:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:43:08.977511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daf48f-e25c-408c-a800-b7d3e6ef1d31 None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:09.008802 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:09.075503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daf48f-e25c-408c-a800-b7d3e6ef1d31 None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:09.107527 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6cde0-3ab8-45bc-bd29-f7b188669c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:09.108114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6cde0-3ab8-45bc-bd29-f7b188669c9a None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:09.108393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6cde0-3ab8-45bc-bd29-f7b188669c9a None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:09.108665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6cde0-3ab8-45bc-bd29-f7b188669c9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:09.109358 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6cde0-3ab8-45bc-bd29-f7b188669c9a None admin] RBAC: Adding query filter params (interface=public, service_id=ce64f32227204dbfba7fb41f9f8695e8, region_id=RegionOne) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:43:09.109662 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6cde0-3ab8-45bc-bd29-f7b188669c9a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:09.111583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6cde0-3ab8-45bc-bd29-f7b188669c9a None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:09.116396 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 647/1300] 10.222.0.55 () {56 vars in 1251 bytes} [Thu Sep 8 13:43:08 2022] GET /identity/v3/endpoints?service_id=ce64f32227204dbfba7fb41f9f8695e8&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:43:09.121908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe0904-a6aa-4e02-a20d-b3a570c1f60a None admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:09.200058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:09.218440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe0904-a6aa-4e02-a20d-b3a570c1f60a None admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:09.251558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b87cd3-7aac-4599-b978-b4b103c71c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:09.252150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b87cd3-7aac-4599-b978-b4b103c71c63 None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:09.252427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b87cd3-7aac-4599-b978-b4b103c71c63 None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:09.252698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b87cd3-7aac-4599-b978-b4b103c71c63 None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:09.253294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b87cd3-7aac-4599-b978-b4b103c71c63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:09.254666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b87cd3-7aac-4599-b978-b4b103c71c63 None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:09.261149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 654/1301] 10.222.0.55 () {56 vars in 1087 bytes} [Thu Sep 8 13:43:09 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:43:11.412547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f4b46-8125-4406-acb8-6437897013ba None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:11.413066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f4b46-8125-4406-acb8-6437897013ba None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:11.413066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f4b46-8125-4406-acb8-6437897013ba None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:11.414450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 648/1302] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:43:11.419700 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1624d6a-dc1f-4f84-a599-0f71e8b6e857 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:11.420079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1624d6a-dc1f-4f84-a599-0f71e8b6e857 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:11.420249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1624d6a-dc1f-4f84-a599-0f71e8b6e857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:11.437985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d1624d6a-dc1f-4f84-a599-0f71e8b6e857 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:43:11.439165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d1624d6a-dc1f-4f84-a599-0f71e8b6e857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:43:11.503229 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 655/1303] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:43:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:43:11.771458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f4b46-8125-4406-acb8-6437897013ba None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:11.863581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:11.870564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f4b46-8125-4406-acb8-6437897013ba None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:11.904777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073a93d-cff8-4a71-a43e-c787523e8c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:11.905451 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073a93d-cff8-4a71-a43e-c787523e8c8e None admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:11.905726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073a93d-cff8-4a71-a43e-c787523e8c8e None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:11.906038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073a93d-cff8-4a71-a43e-c787523e8c8e None admin] PATH_INFO: `/v3/services/placement` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:11.906622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073a93d-cff8-4a71-a43e-c787523e8c8e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:11.907999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073a93d-cff8-4a71-a43e-c787523e8c8e None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application [None req-f073a93d-cff8-4a71-a43e-c787523e8c8e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:43:11.916880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:43:11.918218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application with Lock( Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: ERROR keystone.server.flask.application Sep 08 13:43:11.919097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 649/1304] 10.222.0.55 () {56 vars in 1117 bytes} [Thu Sep 8 13:43:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:43:11.923986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1624d6a-dc1f-4f84-a599-0f71e8b6e857 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:11.997443 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:12.020928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1624d6a-dc1f-4f84-a599-0f71e8b6e857 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:12.053365 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef7819-7b34-4e8c-83bb-6cc323527b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:12.053949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef7819-7b34-4e8c-83bb-6cc323527b2c None admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:12.054229 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef7819-7b34-4e8c-83bb-6cc323527b2c None admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:12.054499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef7819-7b34-4e8c-83bb-6cc323527b2c None admin] PATH_INFO: `/v3/services` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:12.055120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef7819-7b34-4e8c-83bb-6cc323527b2c None admin] RBAC: Adding query filter params (name=placement) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:43:12.055423 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef7819-7b34-4e8c-83bb-6cc323527b2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:12.056781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef7819-7b34-4e8c-83bb-6cc323527b2c None admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:12.061794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 656/1305] 10.222.0.55 () {56 vars in 1116 bytes} [Thu Sep 8 13:43:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:43:12.068270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073a93d-cff8-4a71-a43e-c787523e8c8e None admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:12.164898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073a93d-cff8-4a71-a43e-c787523e8c8e None admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:12.197929 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e56974-39b4-47d6-9c60-d4b062e8b0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:12.198511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e56974-39b4-47d6-9c60-d4b062e8b0c6 None admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:12.198791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e56974-39b4-47d6-9c60-d4b062e8b0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:12.199065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e56974-39b4-47d6-9c60-d4b062e8b0c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:12.199717 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e56974-39b4-47d6-9c60-d4b062e8b0c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:12.201088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e56974-39b4-47d6-9c60-d4b062e8b0c6 None admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:12.219843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 650/1306] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:43:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:43:22.059302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=30 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:22.215449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:17.276773 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9230e-272d-4ed3-89ed-301415ac3bc5 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:17.277251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9230e-272d-4ed3-89ed-301415ac3bc5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:17.277500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9230e-272d-4ed3-89ed-301415ac3bc5 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:17.283867 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 657/1307] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:44:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:44:17.285110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453d330-aef5-4334-ac3a-0791a4a4ef49 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:17.285536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453d330-aef5-4334-ac3a-0791a4a4ef49 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:17.285820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453d330-aef5-4334-ac3a-0791a4a4ef49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:17.387223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b453d330-aef5-4334-ac3a-0791a4a4ef49 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:17.389227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b453d330-aef5-4334-ac3a-0791a4a4ef49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:17.509843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 651/1308] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:44:17.815051 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092ff8d-ebe5-4727-a2ec-73634fea6952 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:17.815411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092ff8d-ebe5-4727-a2ec-73634fea6952 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:17.815619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092ff8d-ebe5-4727-a2ec-73634fea6952 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:17.817125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 658/1309] 10.222.0.55 () {54 vars in 965 bytes} [Thu Sep 8 13:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:44:17.823377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec72e6b-6c38-4802-a26e-24f091560fe0 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:17.823743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec72e6b-6c38-4802-a26e-24f091560fe0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:17.824095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec72e6b-6c38-4802-a26e-24f091560fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:17.923095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1ec72e6b-6c38-4802-a26e-24f091560fe0 None None] MFA Rules not processed for user `531ebc79bf424e9b8762b2c7ec7fe8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:17.924292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1ec72e6b-6c38-4802-a26e-24f091560fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:17.946697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:18.007438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 652/1310] 10.222.0.55 () {58 vars in 1062 bytes} [Thu Sep 8 13:44:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:44:18.016929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092ff8d-ebe5-4727-a2ec-73634fea6952 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:44:18.018695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0092ff8d-ebe5-4727-a2ec-73634fea6952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:18.129635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092ff8d-ebe5-4727-a2ec-73634fea6952 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:44:18.206488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bba5f-4bcd-4e39-bf55-efdba88d3241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:44:18.207095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bba5f-4bcd-4e39-bf55-efdba88d3241 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:18.207373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bba5f-4bcd-4e39-bf55-efdba88d3241 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:18.207643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bba5f-4bcd-4e39-bf55-efdba88d3241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:18.209235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e59bba5f-4bcd-4e39-bf55-efdba88d3241 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:18.268968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:18.326951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bba5f-4bcd-4e39-bf55-efdba88d3241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:44:18.328611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bba5f-4bcd-4e39-bf55-efdba88d3241 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:44:18.348725 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 659/1311] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:44:20.813390 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e40fc-2e50-448c-8f19-aae1a4ac38c3 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:20.813927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e40fc-2e50-448c-8f19-aae1a4ac38c3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:20.813927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e40fc-2e50-448c-8f19-aae1a4ac38c3 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:20.815508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 653/1312] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:44:20.821085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f7e49-7704-4ec8-9666-e80bd1570734 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:20.821506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f7e49-7704-4ec8-9666-e80bd1570734 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:20.821820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f7e49-7704-4ec8-9666-e80bd1570734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:20.901959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-483f7e49-7704-4ec8-9666-e80bd1570734 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:20.903181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-483f7e49-7704-4ec8-9666-e80bd1570734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:21.002297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 660/1313] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:44:21.295118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e40fc-2e50-448c-8f19-aae1a4ac38c3 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:44:21.296719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-657e40fc-2e50-448c-8f19-aae1a4ac38c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:21.402890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e40fc-2e50-448c-8f19-aae1a4ac38c3 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:44:21.474497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:21.476887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06432416-19c6-4b53-b251-f8652ae4cbc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:44:21.477503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06432416-19c6-4b53-b251-f8652ae4cbc9 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:21.478050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06432416-19c6-4b53-b251-f8652ae4cbc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:21.478545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06432416-19c6-4b53-b251-f8652ae4cbc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:21.480386 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-06432416-19c6-4b53-b251-f8652ae4cbc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:21.596699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06432416-19c6-4b53-b251-f8652ae4cbc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:44:21.598161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06432416-19c6-4b53-b251-f8652ae4cbc9 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:44:21.616124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 654/1314] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:44:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:44:24.042131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d8dc6-fe99-453b-928d-123daef48e7c None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:24.042655 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d8dc6-fe99-453b-928d-123daef48e7c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:24.042751 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d8dc6-fe99-453b-928d-123daef48e7c None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:24.044031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 661/1315] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:44:24.049300 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ccb1f0-1558-4eac-bab7-3258e03c800f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:24.049716 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ccb1f0-1558-4eac-bab7-3258e03c800f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:24.049876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ccb1f0-1558-4eac-bab7-3258e03c800f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:24.116483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-37ccb1f0-1558-4eac-bab7-3258e03c800f None None] MFA Rules not processed for user `9fe425c33e1e4fbeb81186f5585399d8`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:24.117703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-37ccb1f0-1558-4eac-bab7-3258e03c800f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:24.168454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:24.212980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 655/1316] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:44:24 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:44:24.498785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d8dc6-fe99-453b-928d-123daef48e7c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:44:24.500327 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-104d8dc6-fe99-453b-928d-123daef48e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:24.510058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:24.602819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d8dc6-fe99-453b-928d-123daef48e7c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:44:24.673780 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b41c2-1064-4dc6-86d0-228085540e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:44:24.674378 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b41c2-1064-4dc6-86d0-228085540e0d service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:24.674655 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b41c2-1064-4dc6-86d0-228085540e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:24.674924 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b41c2-1064-4dc6-86d0-228085540e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:24.676491 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0b0b41c2-1064-4dc6-86d0-228085540e0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:24.738608 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:24.785185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b41c2-1064-4dc6-86d0-228085540e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:44:24.786582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b41c2-1064-4dc6-86d0-228085540e0d service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:44:24.805591 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 662/1317] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:44:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:44:27.497698 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe36faa-bdd7-4d1d-97c0-8b99549e03ca None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:27.498258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe36faa-bdd7-4d1d-97c0-8b99549e03ca None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:27.498258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe36faa-bdd7-4d1d-97c0-8b99549e03ca None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:27.499595 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 656/1318] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:44:27.504908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2c2ea-c332-41aa-b9cd-dd5bf36d04ae None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:27.505339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2c2ea-c332-41aa-b9cd-dd5bf36d04ae None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:27.505445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a2c2ea-c332-41aa-b9cd-dd5bf36d04ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:27.569446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d8a2c2ea-c332-41aa-b9cd-dd5bf36d04ae None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:27.570663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d8a2c2ea-c332-41aa-b9cd-dd5bf36d04ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:27.663632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 663/1319] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:44:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:44:27.952264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe36faa-bdd7-4d1d-97c0-8b99549e03ca None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:44:27.953862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0fe36faa-bdd7-4d1d-97c0-8b99549e03ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:28.062244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe36faa-bdd7-4d1d-97c0-8b99549e03ca None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:44:28.134742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9582f613-ae92-4a99-b5ad-79706b6594aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:44:28.135331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f613-ae92-4a99-b5ad-79706b6594aa service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:28.135608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f613-ae92-4a99-b5ad-79706b6594aa service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:28.135875 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9582f613-ae92-4a99-b5ad-79706b6594aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:28.137454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-9582f613-ae92-4a99-b5ad-79706b6594aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:28.144313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:28.245339 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582f613-ae92-4a99-b5ad-79706b6594aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:44:28.246787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9582f613-ae92-4a99-b5ad-79706b6594aa service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:44:28.265310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 657/1320] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:44:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:44:31.079920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceef8ea-d266-45ca-a0c3-e351d138d0c8 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:31.080450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceef8ea-d266-45ca-a0c3-e351d138d0c8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:31.080450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceef8ea-d266-45ca-a0c3-e351d138d0c8 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:31.081844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 664/1321] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:44:31.086990 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e8ef1-6f0b-47b4-bc81-4c6db4a315cd None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:31.087393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e8ef1-6f0b-47b4-bc81-4c6db4a315cd None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:31.087555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e8ef1-6f0b-47b4-bc81-4c6db4a315cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:31.165283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b90e8ef1-6f0b-47b4-bc81-4c6db4a315cd None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:31.166447 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b90e8ef1-6f0b-47b4-bc81-4c6db4a315cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:31.229613 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:31.272152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 658/1322] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:44:31.558553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceef8ea-d266-45ca-a0c3-e351d138d0c8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:44:31.560168 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-eceef8ea-d266-45ca-a0c3-e351d138d0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:31.580251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:31.663853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceef8ea-d266-45ca-a0c3-e351d138d0c8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:44:31.737355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352fc3f2-6b24-4397-9122-8c3ad6334bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:44:31.737994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fc3f2-6b24-4397-9122-8c3ad6334bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:31.738275 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fc3f2-6b24-4397-9122-8c3ad6334bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:31.738546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352fc3f2-6b24-4397-9122-8c3ad6334bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:31.740136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-352fc3f2-6b24-4397-9122-8c3ad6334bd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:31.811079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:31.849844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352fc3f2-6b24-4397-9122-8c3ad6334bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:44:31.851267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352fc3f2-6b24-4397-9122-8c3ad6334bd4 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:44:31.869490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 665/1323] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:44:34.081944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dfd54-54b9-463d-8013-72c85801fb58 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:34.082470 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dfd54-54b9-463d-8013-72c85801fb58 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:34.082609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076dfd54-54b9-463d-8013-72c85801fb58 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:34.083855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 659/1324] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:44:34.089084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a50ed2-a3fa-4cca-b617-344df014158f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:34.089478 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a50ed2-a3fa-4cca-b617-344df014158f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:34.089647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a50ed2-a3fa-4cca-b617-344df014158f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:34.155752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e1a50ed2-a3fa-4cca-b617-344df014158f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:34.156949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e1a50ed2-a3fa-4cca-b617-344df014158f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:34.265093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 666/1325] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:44:34.552438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dfd54-54b9-463d-8013-72c85801fb58 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:44:34.554100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-076dfd54-54b9-463d-8013-72c85801fb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:34.657758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076dfd54-54b9-463d-8013-72c85801fb58 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:44:34.733175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92602a8a-5744-4c11-a2cd-757148bfb665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:44:34.733805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92602a8a-5744-4c11-a2cd-757148bfb665 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:34.734087 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92602a8a-5744-4c11-a2cd-757148bfb665 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:34.734357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92602a8a-5744-4c11-a2cd-757148bfb665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:34.735947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-92602a8a-5744-4c11-a2cd-757148bfb665 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:34.754047 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:34.852772 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92602a8a-5744-4c11-a2cd-757148bfb665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:44:34.854199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92602a8a-5744-4c11-a2cd-757148bfb665 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:44:34.872083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 660/1326] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:44:37.084221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5681e-e3d3-4ede-b982-e7d779a7b161 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:37.084654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5681e-e3d3-4ede-b982-e7d779a7b161 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:37.084777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5681e-e3d3-4ede-b982-e7d779a7b161 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:37.086371 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 667/1327] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:44:37.091690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd61dc4-faca-4471-ae3b-ff6c3c666189 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:37.092101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd61dc4-faca-4471-ae3b-ff6c3c666189 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:37.092242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd61dc4-faca-4471-ae3b-ff6c3c666189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:37.159693 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-6dd61dc4-faca-4471-ae3b-ff6c3c666189 None None] MFA Rules not processed for user `9fe425c33e1e4fbeb81186f5585399d8`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:37.160896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-6dd61dc4-faca-4471-ae3b-ff6c3c666189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:37.228754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:37.261056 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 661/1328] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:44:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:44:37.546422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5681e-e3d3-4ede-b982-e7d779a7b161 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:44:37.548164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-72a5681e-e3d3-4ede-b982-e7d779a7b161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:37.581469 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:37.655133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5681e-e3d3-4ede-b982-e7d779a7b161 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:44:37.727800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475cd5d-440a-4d2e-a495-78007ec29adb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:44:37.728329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475cd5d-440a-4d2e-a495-78007ec29adb service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:37.728608 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475cd5d-440a-4d2e-a495-78007ec29adb service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:37.728882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475cd5d-440a-4d2e-a495-78007ec29adb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:37.730506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6475cd5d-440a-4d2e-a495-78007ec29adb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:37.805823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:37.838214 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475cd5d-440a-4d2e-a495-78007ec29adb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:44:37.839569 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475cd5d-440a-4d2e-a495-78007ec29adb service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:44:37.857975 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 668/1329] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:44:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:44:40.229063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737b944-24b6-4db3-9042-c7cc6b12d470 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:40.229591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737b944-24b6-4db3-9042-c7cc6b12d470 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:40.229591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737b944-24b6-4db3-9042-c7cc6b12d470 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:40.230881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 662/1330] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:44:40.236017 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8bc89-6a9e-4a41-8cfd-bbbd54e8c1ab None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:40.236459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8bc89-6a9e-4a41-8cfd-bbbd54e8c1ab None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:40.236575 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8bc89-6a9e-4a41-8cfd-bbbd54e8c1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:40.302246 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-cdb8bc89-6a9e-4a41-8cfd-bbbd54e8c1ab None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:40.303436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cdb8bc89-6a9e-4a41-8cfd-bbbd54e8c1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:40.397174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 669/1331] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:44:40.691525 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cae57-07e2-4346-86b8-40f01ccf2d5d None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:40.691803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cae57-07e2-4346-86b8-40f01ccf2d5d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:40.692071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cae57-07e2-4346-86b8-40f01ccf2d5d None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:40.693410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 663/1332] 10.222.0.55 () {54 vars in 965 bytes} [Thu Sep 8 13:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:44:40.699602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330d107-00b5-4452-a505-86549565fca6 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:40.699890 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330d107-00b5-4452-a505-86549565fca6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:40.700165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330d107-00b5-4452-a505-86549565fca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:40.744419 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:40.775994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c330d107-00b5-4452-a505-86549565fca6 None None] MFA Rules not processed for user `531ebc79bf424e9b8762b2c7ec7fe8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:40.777196 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c330d107-00b5-4452-a505-86549565fca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:40.850914 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 670/1333] 10.222.0.55 () {58 vars in 1062 bytes} [Thu Sep 8 13:44:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:44:40.860399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cae57-07e2-4346-86b8-40f01ccf2d5d None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:44:40.861946 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-981cae57-07e2-4346-86b8-40f01ccf2d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:40.962403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cae57-07e2-4346-86b8-40f01ccf2d5d None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:44:41.035628 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0d1148-9ab2-4c2d-9189-c0006825d3bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:44:41.036231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d1148-9ab2-4c2d-9189-c0006825d3bb service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:41.036512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d1148-9ab2-4c2d-9189-c0006825d3bb service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:41.036788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0d1148-9ab2-4c2d-9189-c0006825d3bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:41.038419 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-db0d1148-9ab2-4c2d-9189-c0006825d3bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:41.066008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:41.146319 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d1148-9ab2-4c2d-9189-c0006825d3bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:44:41.147738 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0d1148-9ab2-4c2d-9189-c0006825d3bb service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:44:41.165664 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 664/1334] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:44:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:44:43.568581 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f31050-8e53-48fc-84f2-3fcaabcc210d None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:43.569115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f31050-8e53-48fc-84f2-3fcaabcc210d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:43.569115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f31050-8e53-48fc-84f2-3fcaabcc210d None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:43.570613 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 671/1335] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:44:43.575958 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11626ff4-3324-44c6-b39e-20277009d155 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:43.576361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11626ff4-3324-44c6-b39e-20277009d155 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:43.576501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11626ff4-3324-44c6-b39e-20277009d155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:43.642339 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-11626ff4-3324-44c6-b39e-20277009d155 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:43.643546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-11626ff4-3324-44c6-b39e-20277009d155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:43.725037 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:43.747127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 665/1336] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:44:44.036453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f31050-8e53-48fc-84f2-3fcaabcc210d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:44:44.038004 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b0f31050-8e53-48fc-84f2-3fcaabcc210d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:44.141496 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f31050-8e53-48fc-84f2-3fcaabcc210d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:44:44.169189 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:44.214103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07911b-94a6-4790-9927-575a66ab3b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:44:44.214707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07911b-94a6-4790-9927-575a66ab3b26 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:44.214984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07911b-94a6-4790-9927-575a66ab3b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:44.215255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07911b-94a6-4790-9927-575a66ab3b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:44.216866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5a07911b-94a6-4790-9927-575a66ab3b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:44.324456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07911b-94a6-4790-9927-575a66ab3b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:44:44.325944 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07911b-94a6-4790-9927-575a66ab3b26 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:44:44.346824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 672/1337] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:44:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:44:47.126927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad8364-17e5-4d5e-a33e-4079385fbdbe None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:47.127513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad8364-17e5-4d5e-a33e-4079385fbdbe None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:47.127513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad8364-17e5-4d5e-a33e-4079385fbdbe None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:47.128797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 666/1338] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:44:47.133878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa84c5f-9b67-4395-a482-f6c22e8c3920 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:47.134294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa84c5f-9b67-4395-a482-f6c22e8c3920 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:47.134428 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa84c5f-9b67-4395-a482-f6c22e8c3920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:47.199796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-efa84c5f-9b67-4395-a482-f6c22e8c3920 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:47.200913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-efa84c5f-9b67-4395-a482-f6c22e8c3920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:47.209030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:47.295766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 673/1339] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:44:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:44:47.584354 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad8364-17e5-4d5e-a33e-4079385fbdbe None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:44:47.585934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-bfad8364-17e5-4d5e-a33e-4079385fbdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:47.686533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad8364-17e5-4d5e-a33e-4079385fbdbe None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:44:47.758824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5321b6ec-ae5a-4ab3-9eb9-f56647a4cb92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:44:47.759431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321b6ec-ae5a-4ab3-9eb9-f56647a4cb92 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:47.759703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321b6ec-ae5a-4ab3-9eb9-f56647a4cb92 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:47.759965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5321b6ec-ae5a-4ab3-9eb9-f56647a4cb92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:47.761520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5321b6ec-ae5a-4ab3-9eb9-f56647a4cb92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:47.796222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:47.868661 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321b6ec-ae5a-4ab3-9eb9-f56647a4cb92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:44:47.870125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5321b6ec-ae5a-4ab3-9eb9-f56647a4cb92 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:44:47.888702 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 667/1340] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:44:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:44:52.040210 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902ebc9-6c90-43b7-9712-4a9667eecb37 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:52.040760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902ebc9-6c90-43b7-9712-4a9667eecb37 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:52.040760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902ebc9-6c90-43b7-9712-4a9667eecb37 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:52.042130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 674/1341] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:44:52.047562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d289fa-ad5f-412f-9a18-a0b8e3b0c3b0 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:52.047971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d289fa-ad5f-412f-9a18-a0b8e3b0c3b0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:52.048144 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d289fa-ad5f-412f-9a18-a0b8e3b0c3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:52.114394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-96d289fa-ad5f-412f-9a18-a0b8e3b0c3b0 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:52.115603 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-96d289fa-ad5f-412f-9a18-a0b8e3b0c3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:52.206667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:52.219032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 668/1342] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:44:52.514481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902ebc9-6c90-43b7-9712-4a9667eecb37 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:44:52.516018 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b902ebc9-6c90-43b7-9712-4a9667eecb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:52.625203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902ebc9-6c90-43b7-9712-4a9667eecb37 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:44:52.662189 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:52.698579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee187143-d1a8-4d82-aeab-3c34befa5d93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:44:52.699180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee187143-d1a8-4d82-aeab-3c34befa5d93 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:52.699495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee187143-d1a8-4d82-aeab-3c34befa5d93 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:52.699765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee187143-d1a8-4d82-aeab-3c34befa5d93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:52.701331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ee187143-d1a8-4d82-aeab-3c34befa5d93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:52.807996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee187143-d1a8-4d82-aeab-3c34befa5d93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:44:52.809942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee187143-d1a8-4d82-aeab-3c34befa5d93 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:44:52.827957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 675/1343] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:44:55.635562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2a00a-c082-410d-824f-367124b7c571 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:55.636128 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2a00a-c082-410d-824f-367124b7c571 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:55.636128 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2a00a-c082-410d-824f-367124b7c571 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:55.637460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 669/1344] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:44:55.642723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7902c-2cd1-48f8-9b35-94c90b8880e1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:55.643169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7902c-2cd1-48f8-9b35-94c90b8880e1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:55.643268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7902c-2cd1-48f8-9b35-94c90b8880e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:55.708430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9bf7902c-2cd1-48f8-9b35-94c90b8880e1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:44:55.709656 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9bf7902c-2cd1-48f8-9b35-94c90b8880e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:55.727642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:55.808344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 676/1345] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:44:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:44:56.091725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2a00a-c082-410d-824f-367124b7c571 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:44:56.093261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7cc2a00a-c082-410d-824f-367124b7c571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:56.204717 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2a00a-c082-410d-824f-367124b7c571 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:44:56.278297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2211aa-e60b-4216-8643-4275f5fbe437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:44:56.278903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2211aa-e60b-4216-8643-4275f5fbe437 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:44:56.279180 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2211aa-e60b-4216-8643-4275f5fbe437 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:44:56.279448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2211aa-e60b-4216-8643-4275f5fbe437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:44:56.281062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-dd2211aa-e60b-4216-8643-4275f5fbe437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:44:56.325565 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:44:56.399532 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2211aa-e60b-4216-8643-4275f5fbe437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:44:56.400954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2211aa-e60b-4216-8643-4275f5fbe437 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:44:56.420180 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 670/1346] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:44:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:00.584053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3132335-f477-4d85-8423-a7d347d64bdd None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:00.584524 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3132335-f477-4d85-8423-a7d347d64bdd None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:00.584620 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3132335-f477-4d85-8423-a7d347d64bdd None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:00.589778 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 677/1347] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:00.591741 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411a97f-33ee-4662-956f-d31c0d8e8b9c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:00.592258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411a97f-33ee-4662-956f-d31c0d8e8b9c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:00.592258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411a97f-33ee-4662-956f-d31c0d8e8b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:00.667829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-0411a97f-33ee-4662-956f-d31c0d8e8b9c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:00.668974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0411a97f-33ee-4662-956f-d31c0d8e8b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:00.781123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:00.783815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 671/1348] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:01.070764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3132335-f477-4d85-8423-a7d347d64bdd None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:01.072340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c3132335-f477-4d85-8423-a7d347d64bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:01.173530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3132335-f477-4d85-8423-a7d347d64bdd None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:01.221075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:01.246458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a4c58-7509-4f47-834a-523dd7cc1733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:01.247079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a4c58-7509-4f47-834a-523dd7cc1733 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:01.247354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a4c58-7509-4f47-834a-523dd7cc1733 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:01.247622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a4c58-7509-4f47-834a-523dd7cc1733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:01.249179 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-103a4c58-7509-4f47-834a-523dd7cc1733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:01.357571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a4c58-7509-4f47-834a-523dd7cc1733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:01.358945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a4c58-7509-4f47-834a-523dd7cc1733 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:01.377212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 678/1349] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:06.287767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2fc46-797d-489d-96bc-ec1418b1dd59 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:06.288281 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2fc46-797d-489d-96bc-ec1418b1dd59 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:06.288401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2fc46-797d-489d-96bc-ec1418b1dd59 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:06.290166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 672/1350] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 13:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:06.295966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad422a39-3024-4ca7-a87f-03b54acbc25b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:06.295966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad422a39-3024-4ca7-a87f-03b54acbc25b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:06.296415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad422a39-3024-4ca7-a87f-03b54acbc25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:06.415359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ad422a39-3024-4ca7-a87f-03b54acbc25b None None] MFA Rules not processed for user `e42fb05dbae64c758098c591d6ffa790`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:06.416895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ad422a39-3024-4ca7-a87f-03b54acbc25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:06.461653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:06.572337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 679/1351] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 13:45:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:10.682508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20316e1b-e4db-4b8a-b8b2-9d973228f50a None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:10.683124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20316e1b-e4db-4b8a-b8b2-9d973228f50a None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:10.683959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20316e1b-e4db-4b8a-b8b2-9d973228f50a None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:10.685415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 673/1352] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 13:45:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:10.691410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4847d92-fbaa-4ab1-b480-809087b8a969 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:10.691827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4847d92-fbaa-4ab1-b480-809087b8a969 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:10.691955 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4847d92-fbaa-4ab1-b480-809087b8a969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:10.771408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b4847d92-fbaa-4ab1-b480-809087b8a969 None None] MFA Rules not processed for user `e42fb05dbae64c758098c591d6ffa790`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:10.772594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b4847d92-fbaa-4ab1-b480-809087b8a969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:10.850742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:10.888667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 680/1353] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 13:45:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:12.416165 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47a03e-e3ab-4c0e-87fd-93d6671b4ffe None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:12.416746 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47a03e-e3ab-4c0e-87fd-93d6671b4ffe None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:12.416746 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47a03e-e3ab-4c0e-87fd-93d6671b4ffe None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:12.418870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 674/1354] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 13:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:12.423635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e96883-a7f7-4b63-97cf-0baed999ef25 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:12.424058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e96883-a7f7-4b63-97cf-0baed999ef25 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:12.424180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e96883-a7f7-4b63-97cf-0baed999ef25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:12.503675 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f0e96883-a7f7-4b63-97cf-0baed999ef25 None None] MFA Rules not processed for user `e42fb05dbae64c758098c591d6ffa790`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:12.504866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f0e96883-a7f7-4b63-97cf-0baed999ef25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:12.613328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 681/1355] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 13:45:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:16.101937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f30b-d328-4b62-9cf9-bc1c27fe1920 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:16.102455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f30b-d328-4b62-9cf9-bc1c27fe1920 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:16.102676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f30b-d328-4b62-9cf9-bc1c27fe1920 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:16.104139 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 675/1356] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:16.109581 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a08b77-8060-421c-ac36-8f264ad80ca6 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:16.110097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a08b77-8060-421c-ac36-8f264ad80ca6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:16.110184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a08b77-8060-421c-ac36-8f264ad80ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:16.150182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:16.183574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a2a08b77-8060-421c-ac36-8f264ad80ca6 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:16.184767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a2a08b77-8060-421c-ac36-8f264ad80ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:16.292961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 682/1357] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:45:16.601765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f4c46-44cc-4d8c-8a29-8ccfe60956f8 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:16.602257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f4c46-44cc-4d8c-8a29-8ccfe60956f8 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:16.602748 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350f4c46-44cc-4d8c-8a29-8ccfe60956f8 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:16.605143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 676/1358] 10.222.0.55 () {54 vars in 962 bytes} [Thu Sep 8 13:45:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:16.614031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35648f80-bb9e-46ff-a5c0-db41fd889fda None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:16.614488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35648f80-bb9e-46ff-a5c0-db41fd889fda None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:16.616489 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35648f80-bb9e-46ff-a5c0-db41fd889fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:16.706833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-35648f80-bb9e-46ff-a5c0-db41fd889fda None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:16.708213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-35648f80-bb9e-46ff-a5c0-db41fd889fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:16.761382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:16.839318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 683/1359] 10.222.0.55 () {58 vars in 1059 bytes} [Thu Sep 8 13:45:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:45:16.847076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f4c46-44cc-4d8c-8a29-8ccfe60956f8 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:16.848705 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-350f4c46-44cc-4d8c-8a29-8ccfe60956f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:17.016421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350f4c46-44cc-4d8c-8a29-8ccfe60956f8 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:17.100922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:17.145512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d75bc-5e59-474a-9847-08e113d1933c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:17.146493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d75bc-5e59-474a-9847-08e113d1933c service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:17.146781 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d75bc-5e59-474a-9847-08e113d1933c service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:17.147058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d75bc-5e59-474a-9847-08e113d1933c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:17.148658 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-fa8d75bc-5e59-474a-9847-08e113d1933c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:17.251468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d75bc-5e59-474a-9847-08e113d1933c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:17.253243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d75bc-5e59-474a-9847-08e113d1933c service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:17.273021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 677/1360] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:19.689851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8143c9-e0df-468f-9899-dae7c2a6526c None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:19.689851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8143c9-e0df-468f-9899-dae7c2a6526c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:19.689851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8143c9-e0df-468f-9899-dae7c2a6526c None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:19.689851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 684/1361] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:19.694186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec3f87-1fae-4011-a1eb-ed6b48f21f25 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:19.694186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec3f87-1fae-4011-a1eb-ed6b48f21f25 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:19.694671 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec3f87-1fae-4011-a1eb-ed6b48f21f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:19.766816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-33ec3f87-1fae-4011-a1eb-ed6b48f21f25 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:19.768020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-33ec3f87-1fae-4011-a1eb-ed6b48f21f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:19.809273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:19.878193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 678/1362] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:20.169520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8143c9-e0df-468f-9899-dae7c2a6526c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:20.171170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1e8143c9-e0df-468f-9899-dae7c2a6526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:20.350067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8143c9-e0df-468f-9899-dae7c2a6526c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:20.360884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:20.486830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b068b-a2e6-4445-aee0-75164f4f2523 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:20.487442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b068b-a2e6-4445-aee0-75164f4f2523 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:20.487724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b068b-a2e6-4445-aee0-75164f4f2523 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:20.487998 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b068b-a2e6-4445-aee0-75164f4f2523 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:20.489608 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fe0b068b-a2e6-4445-aee0-75164f4f2523 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:20.583969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b068b-a2e6-4445-aee0-75164f4f2523 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:20.585571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b068b-a2e6-4445-aee0-75164f4f2523 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:20.603979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 685/1363] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:23.079027 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729f830-be44-4c12-92aa-a93dc5e337e9 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:23.079937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729f830-be44-4c12-92aa-a93dc5e337e9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:23.080394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729f830-be44-4c12-92aa-a93dc5e337e9 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:23.081933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 679/1364] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:23.087172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bd25f-2de8-4714-95b0-87c04ebb4d7c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:23.087892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bd25f-2de8-4714-95b0-87c04ebb4d7c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:23.088313 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949bd25f-2de8-4714-95b0-87c04ebb4d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:23.105330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:23.156783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-949bd25f-2de8-4714-95b0-87c04ebb4d7c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:23.158004 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-949bd25f-2de8-4714-95b0-87c04ebb4d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:23.258846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 686/1365] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:23.545604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f4b5e-9afc-4ceb-b32d-ecdfc57495c9 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:23.546121 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f4b5e-9afc-4ceb-b32d-ecdfc57495c9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:23.546499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f4b5e-9afc-4ceb-b32d-ecdfc57495c9 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:23.548095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 680/1366] 10.222.0.55 () {54 vars in 962 bytes} [Thu Sep 8 13:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:23.553622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667823f-c5ed-4145-aa15-4484163dec8a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:23.554088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667823f-c5ed-4145-aa15-4484163dec8a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:23.554520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667823f-c5ed-4145-aa15-4484163dec8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:23.636563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8667823f-c5ed-4145-aa15-4484163dec8a None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:23.638073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8667823f-c5ed-4145-aa15-4484163dec8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:23.674526 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:23.762022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 687/1367] 10.222.0.55 () {58 vars in 1059 bytes} [Thu Sep 8 13:45:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:23.767249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f4b5e-9afc-4ceb-b32d-ecdfc57495c9 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:23.768872 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-243f4b5e-9afc-4ceb-b32d-ecdfc57495c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:23.934709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f4b5e-9afc-4ceb-b32d-ecdfc57495c9 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:23.956834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:24.061240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d1188-a001-445f-8cc9-acf97bccf01f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:24.061974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d1188-a001-445f-8cc9-acf97bccf01f service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:24.062354 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d1188-a001-445f-8cc9-acf97bccf01f service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:24.062725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d1188-a001-445f-8cc9-acf97bccf01f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:24.064729 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7e1d1188-a001-445f-8cc9-acf97bccf01f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:24.163188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d1188-a001-445f-8cc9-acf97bccf01f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:24.165024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1d1188-a001-445f-8cc9-acf97bccf01f service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:24.185996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 681/1368] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:45:24.265010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb092c01-d087-4592-9b87-3b213ade5e27 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:24.265531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb092c01-d087-4592-9b87-3b213ade5e27 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:24.265531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb092c01-d087-4592-9b87-3b213ade5e27 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:24.267096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 688/1369] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:24.272283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f5caa-e9d2-4157-9b98-a97ae8133d02 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:24.272562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f5caa-e9d2-4157-9b98-a97ae8133d02 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:24.272835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f5caa-e9d2-4157-9b98-a97ae8133d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:24.307399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:24.355953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-622f5caa-e9d2-4157-9b98-a97ae8133d02 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:24.357253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-622f5caa-e9d2-4157-9b98-a97ae8133d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:24.480204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 682/1370] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:24.811626 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157c06e-65c8-4249-af5b-481878b974bd None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:24.811626 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157c06e-65c8-4249-af5b-481878b974bd None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:24.811626 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157c06e-65c8-4249-af5b-481878b974bd None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:24.811626 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 689/1371] 10.222.0.55 () {54 vars in 964 bytes} [Thu Sep 8 13:45:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:24.816972 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64410865-5422-470e-85d9-09c1c77944c8 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:24.817282 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64410865-5422-470e-85d9-09c1c77944c8 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:24.817552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64410865-5422-470e-85d9-09c1c77944c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:24.900871 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-64410865-5422-470e-85d9-09c1c77944c8 None None] MFA Rules not processed for user `ff93243f0c4c48d1924ea04a87a0e7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:24.902153 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-64410865-5422-470e-85d9-09c1c77944c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:24.959785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:24.992389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 683/1372] 10.222.0.55 () {58 vars in 1061 bytes} [Thu Sep 8 13:45:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:25.000430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157c06e-65c8-4249-af5b-481878b974bd None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:25.001976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1157c06e-65c8-4249-af5b-481878b974bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:25.111535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157c06e-65c8-4249-af5b-481878b974bd None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:25.158378 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:25.202310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438f741-3b95-4f7f-b92e-be11d0c2fa97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:25.203008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438f741-3b95-4f7f-b92e-be11d0c2fa97 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:25.203294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438f741-3b95-4f7f-b92e-be11d0c2fa97 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:25.203557 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438f741-3b95-4f7f-b92e-be11d0c2fa97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:25.205177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a438f741-3b95-4f7f-b92e-be11d0c2fa97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:25.330264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438f741-3b95-4f7f-b92e-be11d0c2fa97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:25.331609 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438f741-3b95-4f7f-b92e-be11d0c2fa97 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:25.352546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 690/1373] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:26.468916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae8455-3aa6-40c2-8a4e-f1ca25141ba6 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:26.469531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae8455-3aa6-40c2-8a4e-f1ca25141ba6 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:26.469531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae8455-3aa6-40c2-8a4e-f1ca25141ba6 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:26.470945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 684/1374] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:26.476255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ec64f-9be0-4e27-b920-39cacbb6d53e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:26.476653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ec64f-9be0-4e27-b920-39cacbb6d53e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:26.476812 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ec64f-9be0-4e27-b920-39cacbb6d53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:26.542609 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-df8ec64f-9be0-4e27-b920-39cacbb6d53e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:26.543801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-df8ec64f-9be0-4e27-b920-39cacbb6d53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:26.563996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:26.640176 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 691/1375] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:26.936902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae8455-3aa6-40c2-8a4e-f1ca25141ba6 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:26.938745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-efae8455-3aa6-40c2-8a4e-f1ca25141ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:27.089726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae8455-3aa6-40c2-8a4e-f1ca25141ba6 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:27.143712 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:27.210206 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1a0b6-7f8c-49e1-9bda-fdd1d1de243f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:27.210816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1a0b6-7f8c-49e1-9bda-fdd1d1de243f service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:27.211100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1a0b6-7f8c-49e1-9bda-fdd1d1de243f service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:27.211368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1a0b6-7f8c-49e1-9bda-fdd1d1de243f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:27.212970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-64b1a0b6-7f8c-49e1-9bda-fdd1d1de243f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:27.306795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1a0b6-7f8c-49e1-9bda-fdd1d1de243f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:27.308970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1a0b6-7f8c-49e1-9bda-fdd1d1de243f service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:27.328171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 685/1376] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:29.521851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50687877-7e2c-450f-b775-a82f66b790c1 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:29.522413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50687877-7e2c-450f-b775-a82f66b790c1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:29.522413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50687877-7e2c-450f-b775-a82f66b790c1 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:29.523901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 692/1377] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:29.529621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093c774-0f7e-4284-a2e0-6e8de44fc5d2 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:29.530078 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093c774-0f7e-4284-a2e0-6e8de44fc5d2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:29.530304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f093c774-0f7e-4284-a2e0-6e8de44fc5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:29.603096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-f093c774-0f7e-4284-a2e0-6e8de44fc5d2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:29.604309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f093c774-0f7e-4284-a2e0-6e8de44fc5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:29.611880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:29.715533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 686/1378] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:45:30.008219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50687877-7e2c-450f-b775-a82f66b790c1 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:30.009715 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-50687877-7e2c-450f-b775-a82f66b790c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:30.161825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50687877-7e2c-450f-b775-a82f66b790c1 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:30.165831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:30.280908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fef19-48df-48ab-9216-a3af56ddf225 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:30.281516 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fef19-48df-48ab-9216-a3af56ddf225 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:30.281817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fef19-48df-48ab-9216-a3af56ddf225 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:30.282090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fef19-48df-48ab-9216-a3af56ddf225 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:30.283653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cd8fef19-48df-48ab-9216-a3af56ddf225 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:30.377866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fef19-48df-48ab-9216-a3af56ddf225 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:30.379442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fef19-48df-48ab-9216-a3af56ddf225 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:30.397812 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 693/1379] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:32.624126 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378e888-b257-4580-8ce7-6e5e47cccb5c None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:32.624639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378e888-b257-4580-8ce7-6e5e47cccb5c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:32.624639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378e888-b257-4580-8ce7-6e5e47cccb5c None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:32.626253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 687/1380] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:32.631430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89104d69-6db9-4eae-b5ed-aeb9c3a40a40 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:32.631853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89104d69-6db9-4eae-b5ed-aeb9c3a40a40 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:32.632082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89104d69-6db9-4eae-b5ed-aeb9c3a40a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:32.643565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:32.702954 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-89104d69-6db9-4eae-b5ed-aeb9c3a40a40 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:32.704138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-89104d69-6db9-4eae-b5ed-aeb9c3a40a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:32.814061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 694/1381] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:33.124759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378e888-b257-4580-8ce7-6e5e47cccb5c None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:33.126806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2378e888-b257-4580-8ce7-6e5e47cccb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:33.281534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:33.294329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378e888-b257-4580-8ce7-6e5e47cccb5c None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:33.415002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66baa93-76e0-4313-9b06-bae2a81bab6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:33.415610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66baa93-76e0-4313-9b06-bae2a81bab6e service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:33.415885 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66baa93-76e0-4313-9b06-bae2a81bab6e service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:33.416176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66baa93-76e0-4313-9b06-bae2a81bab6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:33.417811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f66baa93-76e0-4313-9b06-bae2a81bab6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:33.527434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66baa93-76e0-4313-9b06-bae2a81bab6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:33.529042 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66baa93-76e0-4313-9b06-bae2a81bab6e service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:33.538899 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:33.548624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 688/1382] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:36.008583 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95e400-5297-4d4d-b095-d2c7763e2815 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:36.009156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95e400-5297-4d4d-b095-d2c7763e2815 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:36.009156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95e400-5297-4d4d-b095-d2c7763e2815 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:36.010484 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 695/1383] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:36.017082 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953b8d3-d992-4692-be3d-2a92f1839c1d None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:36.017696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953b8d3-d992-4692-be3d-2a92f1839c1d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:36.017843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953b8d3-d992-4692-be3d-2a92f1839c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:36.095490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1953b8d3-d992-4692-be3d-2a92f1839c1d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:36.097239 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1953b8d3-d992-4692-be3d-2a92f1839c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:36.202756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 689/1384] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:45:36.509399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95e400-5297-4d4d-b095-d2c7763e2815 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:36.511487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5a95e400-5297-4d4d-b095-d2c7763e2815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:36.604196 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:36.678447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95e400-5297-4d4d-b095-d2c7763e2815 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:36.802095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcfc410-0090-4725-a4a4-22e9ada8f780 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:36.802703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfc410-0090-4725-a4a4-22e9ada8f780 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:36.802979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfc410-0090-4725-a4a4-22e9ada8f780 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:36.803244 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcfc410-0090-4725-a4a4-22e9ada8f780 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:36.804822 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9dcfc410-0090-4725-a4a4-22e9ada8f780 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:36.859101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:36.906939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfc410-0090-4725-a4a4-22e9ada8f780 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:36.906939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcfc410-0090-4725-a4a4-22e9ada8f780 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:36.924938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 696/1385] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:39.421719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cf52d-8da1-4b48-99b9-b8d860def7e1 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:39.422253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cf52d-8da1-4b48-99b9-b8d860def7e1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:39.422342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cf52d-8da1-4b48-99b9-b8d860def7e1 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:39.423607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 690/1386] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:39.429311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73587d3e-8421-49e9-9179-213e0f7ef446 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:39.429756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73587d3e-8421-49e9-9179-213e0f7ef446 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:39.430212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73587d3e-8421-49e9-9179-213e0f7ef446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:39.496681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-73587d3e-8421-49e9-9179-213e0f7ef446 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:39.497916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-73587d3e-8421-49e9-9179-213e0f7ef446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:39.605652 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 697/1387] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:39.898785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03cf52d-8da1-4b48-99b9-b8d860def7e1 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:39.900355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e03cf52d-8da1-4b48-99b9-b8d860def7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:39.977052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:40.048771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03cf52d-8da1-4b48-99b9-b8d860def7e1 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:40.163463 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec25448-dec9-4591-b243-61c38c7bafb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:40.163995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec25448-dec9-4591-b243-61c38c7bafb0 service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:40.164276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec25448-dec9-4591-b243-61c38c7bafb0 service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:40.164548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec25448-dec9-4591-b243-61c38c7bafb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:40.166173 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2ec25448-dec9-4591-b243-61c38c7bafb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:40.206680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:40.259246 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec25448-dec9-4591-b243-61c38c7bafb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:40.261388 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec25448-dec9-4591-b243-61c38c7bafb0 service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:40.279828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 691/1388] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:40.650525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce5ef3-c129-4597-960f-5c2ac20c644d None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:40.651558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce5ef3-c129-4597-960f-5c2ac20c644d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:40.651558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce5ef3-c129-4597-960f-5c2ac20c644d None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:40.652908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 698/1389] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:40.658319 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc7615-a562-4097-b38f-9e34eb62f0e0 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:40.658719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc7615-a562-4097-b38f-9e34eb62f0e0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:40.658873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc7615-a562-4097-b38f-9e34eb62f0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:40.726915 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-7dbc7615-a562-4097-b38f-9e34eb62f0e0 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:40.728112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7dbc7615-a562-4097-b38f-9e34eb62f0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:40.827188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 692/1390] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:45:41.138873 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c3676-4c1e-4473-9489-c0aedf2ffe30 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:41.139164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c3676-4c1e-4473-9489-c0aedf2ffe30 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:41.139443 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c3676-4c1e-4473-9489-c0aedf2ffe30 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:41.141076 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 699/1391] 10.222.0.55 () {54 vars in 961 bytes} [Thu Sep 8 13:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:41.146234 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cb15f-1930-4e7b-a322-4f10e0704e4e None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:41.147193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cb15f-1930-4e7b-a322-4f10e0704e4e None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:41.147193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915cb15f-1930-4e7b-a322-4f10e0704e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:41.163153 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:41.230851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-915cb15f-1930-4e7b-a322-4f10e0704e4e None None] MFA Rules not processed for user `4e27f5b81c704dbebbd6a83b2713e237`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:41.232061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-915cb15f-1930-4e7b-a322-4f10e0704e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:41.308357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 693/1392] 10.222.0.55 () {58 vars in 1058 bytes} [Thu Sep 8 13:45:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:41.317949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c3676-4c1e-4473-9489-c0aedf2ffe30 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:41.319475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-911c3676-4c1e-4473-9489-c0aedf2ffe30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:41.349588 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:41.424122 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c3676-4c1e-4473-9489-c0aedf2ffe30 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:41.499686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62796e-0e87-4b84-be8d-397461c5bde8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e27f5b81c704dbebbd6a83b2713e237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:41.500303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62796e-0e87-4b84-be8d-397461c5bde8 service glance] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:41.500584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62796e-0e87-4b84-be8d-397461c5bde8 service glance] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:41.500859 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62796e-0e87-4b84-be8d-397461c5bde8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:41.502490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cd62796e-0e87-4b84-be8d-397461c5bde8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:41.581825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:41.615603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62796e-0e87-4b84-be8d-397461c5bde8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:41.617050 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62796e-0e87-4b84-be8d-397461c5bde8 service glance] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:41.635834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 700/1393] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:42.397464 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e8b1d-9a9e-46c4-9558-cdc00c961055 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:42.398047 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e8b1d-9a9e-46c4-9558-cdc00c961055 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:42.398221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e8b1d-9a9e-46c4-9558-cdc00c961055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:42.474441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-025e8b1d-9a9e-46c4-9558-cdc00c961055 None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:42.475911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-025e8b1d-9a9e-46c4-9558-cdc00c961055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:42.508250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bc300-7887-453a-9cbc-bad910360326 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:42.508768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bc300-7887-453a-9cbc-bad910360326 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:42.508768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bc300-7887-453a-9cbc-bad910360326 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:42.510198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 701/1394] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:45:42.515054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74e33a-1bc3-4810-b2dc-5ae84ec4bee4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:42.515342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74e33a-1bc3-4810-b2dc-5ae84ec4bee4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:42.515617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74e33a-1bc3-4810-b2dc-5ae84ec4bee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:42.532585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:42.566144 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 694/1395] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 13:45:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:42.582543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1b74e33a-1bc3-4810-b2dc-5ae84ec4bee4 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:42.583758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1b74e33a-1bc3-4810-b2dc-5ae84ec4bee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:42.616055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935076e6-af18-4548-9870-b30dab0b8920 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:42.616420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935076e6-af18-4548-9870-b30dab0b8920 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:42.616763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935076e6-af18-4548-9870-b30dab0b8920 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:42.618258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 695/1396] 10.222.0.55 () {54 vars in 943 bytes} [Thu Sep 8 13:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:45:42.624567 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddea81c-f956-442b-af85-00fabe3a6c2f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:42.624918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddea81c-f956-442b-af85-00fabe3a6c2f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:42.625596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddea81c-f956-442b-af85-00fabe3a6c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:42.684694 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 702/1397] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:45:42.705542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-cddea81c-f956-442b-af85-00fabe3a6c2f None None] MFA Rules not processed for user `4e5d73e8daa84975bc21e41e3cbef9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:42.706856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-cddea81c-f956-442b-af85-00fabe3a6c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:42.814225 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 696/1398] 10.222.0.55 () {58 vars in 1040 bytes} [Thu Sep 8 13:45:42 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:45:42.824086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74e33a-1bc3-4810-b2dc-5ae84ec4bee4 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:42.825295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1b74e33a-1bc3-4810-b2dc-5ae84ec4bee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:42.864240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:42.934762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74e33a-1bc3-4810-b2dc-5ae84ec4bee4 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:42.981954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddea81c-f956-442b-af85-00fabe3a6c2f None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:42.983158 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-cddea81c-f956-442b-af85-00fabe3a6c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:43.017305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:43.040290 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd08372-0d41-4872-943e-ce030fae51ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:43.040960 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd08372-0d41-4872-943e-ce030fae51ed service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:43.041763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd08372-0d41-4872-943e-ce030fae51ed service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:43.042039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd08372-0d41-4872-943e-ce030fae51ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:43.043699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4fd08372-0d41-4872-943e-ce030fae51ed service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:43.085069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddea81c-f956-442b-af85-00fabe3a6c2f None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:43.098669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:43.129166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd08372-0d41-4872-943e-ce030fae51ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:43.130731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd08372-0d41-4872-943e-ce030fae51ed service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:43.138852 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 703/1399] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 13:45:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:45:43.225129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6f5dd-8dbb-4f98-89d1-3db978f53b8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:43.225947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6f5dd-8dbb-4f98-89d1-3db978f53b8e service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:43.226343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6f5dd-8dbb-4f98-89d1-3db978f53b8e service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:43.226455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6f5dd-8dbb-4f98-89d1-3db978f53b8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:43.228221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5dd6f5dd-8dbb-4f98-89d1-3db978f53b8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:43.272864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:43.329652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6f5dd-8dbb-4f98-89d1-3db978f53b8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:43.331295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6f5dd-8dbb-4f98-89d1-3db978f53b8e service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:43.350677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 697/1400] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:45.571199 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6464a-abdd-4412-966d-19dd8bfcce13 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:45.571770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6464a-abdd-4412-966d-19dd8bfcce13 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:45.571770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6464a-abdd-4412-966d-19dd8bfcce13 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:45.573605 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 704/1401] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:45.578939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358894c7-c0cf-4a7d-b411-b7a56606f3dd None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:45.579351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358894c7-c0cf-4a7d-b411-b7a56606f3dd None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:45.579532 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358894c7-c0cf-4a7d-b411-b7a56606f3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:45.659044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-358894c7-c0cf-4a7d-b411-b7a56606f3dd None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:45.660257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-358894c7-c0cf-4a7d-b411-b7a56606f3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:45.766316 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 698/1402] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:45:46.069619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6464a-abdd-4412-966d-19dd8bfcce13 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:46.071665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5ca6464a-abdd-4412-966d-19dd8bfcce13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:46.237928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6464a-abdd-4412-966d-19dd8bfcce13 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:46.293984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:46.391487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82777526-107d-468f-b49f-13d3824678a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:46.392222 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82777526-107d-468f-b49f-13d3824678a2 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:46.392717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82777526-107d-468f-b49f-13d3824678a2 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:46.393063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82777526-107d-468f-b49f-13d3824678a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:46.394992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-82777526-107d-468f-b49f-13d3824678a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:46.589290 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82777526-107d-468f-b49f-13d3824678a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:46.590925 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82777526-107d-468f-b49f-13d3824678a2 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:46.611358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 705/1403] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:48.837075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59adae-04bb-45c7-aa75-eeeac04d3e3a None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:48.838141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59adae-04bb-45c7-aa75-eeeac04d3e3a None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:48.838141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59adae-04bb-45c7-aa75-eeeac04d3e3a None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:48.839468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 699/1404] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:48.845044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5363c9e-7a1e-4b9b-853d-e37261762929 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:48.845596 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5363c9e-7a1e-4b9b-853d-e37261762929 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:48.845596 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5363c9e-7a1e-4b9b-853d-e37261762929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:48.921271 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d5363c9e-7a1e-4b9b-853d-e37261762929 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:48.922559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d5363c9e-7a1e-4b9b-853d-e37261762929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:48.925513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:49.037926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 706/1405] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:45:49.328005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59adae-04bb-45c7-aa75-eeeac04d3e3a None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:49.329519 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4b59adae-04bb-45c7-aa75-eeeac04d3e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:49.351573 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:49.505976 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59adae-04bb-45c7-aa75-eeeac04d3e3a None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:49.607606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:49.632321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62a944-3e2e-4037-916a-4b956380a913 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:49.633037 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62a944-3e2e-4037-916a-4b956380a913 service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:49.633315 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62a944-3e2e-4037-916a-4b956380a913 service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:49.633580 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62a944-3e2e-4037-916a-4b956380a913 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:49.635228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8d62a944-3e2e-4037-916a-4b956380a913 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:49.732260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62a944-3e2e-4037-916a-4b956380a913 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:49.733907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d62a944-3e2e-4037-916a-4b956380a913 service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:49.752200 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 700/1406] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:45:51.969452 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d48e6c-b70d-4340-b544-4113de9fb2d2 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:51.970065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d48e6c-b70d-4340-b544-4113de9fb2d2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:51.970065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d48e6c-b70d-4340-b544-4113de9fb2d2 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:51.971357 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 707/1407] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:45:51.976610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7326b5-2e7e-4bc7-8535-c598e9ed2611 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:51.977054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7326b5-2e7e-4bc7-8535-c598e9ed2611 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:51.977159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7326b5-2e7e-4bc7-8535-c598e9ed2611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:52.060045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1b7326b5-2e7e-4bc7-8535-c598e9ed2611 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:45:52.061265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1b7326b5-2e7e-4bc7-8535-c598e9ed2611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:52.118752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:52.172139 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 701/1408] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:45:52.459817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d48e6c-b70d-4340-b544-4113de9fb2d2 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:45:52.461515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e3d48e6c-b70d-4340-b544-4113de9fb2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:52.622344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:52.635525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d48e6c-b70d-4340-b544-4113de9fb2d2 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:45:52.757880 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e438f-c39d-46da-92ad-0f1605c3dee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:45:52.758406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e438f-c39d-46da-92ad-0f1605c3dee0 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:45:52.758826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e438f-c39d-46da-92ad-0f1605c3dee0 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:45:52.759122 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e438f-c39d-46da-92ad-0f1605c3dee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:45:52.761263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-430e438f-c39d-46da-92ad-0f1605c3dee0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:45:52.862538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e438f-c39d-46da-92ad-0f1605c3dee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:45:52.864516 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e438f-c39d-46da-92ad-0f1605c3dee0 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:45:52.874684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:45:52.885777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 708/1409] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:45:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:02.168414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=19 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:02.882794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:06.075164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c3bae-809b-4763-8910-08a486e26ae8 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:06.075747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c3bae-809b-4763-8910-08a486e26ae8 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:06.075747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c3bae-809b-4763-8910-08a486e26ae8 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:06.077055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 702/1410] 10.222.0.55 () {54 vars in 918 bytes} [Thu Sep 8 13:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:06.082240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b4681-197e-48e3-b8ea-05b37e41e289 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:06.082668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b4681-197e-48e3-b8ea-05b37e41e289 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:06.082817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138b4681-197e-48e3-b8ea-05b37e41e289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:06.157405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-138b4681-197e-48e3-b8ea-05b37e41e289 None None] MFA Rules not processed for user `e42fb05dbae64c758098c591d6ffa790`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:06.158622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-138b4681-197e-48e3-b8ea-05b37e41e289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:06.264635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 709/1411] 10.222.0.55 () {58 vars in 1015 bytes} [Thu Sep 8 13:46:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:06.279226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829f8a0-aa70-4be7-9872-f74af49249bd None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:06.279511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829f8a0-aa70-4be7-9872-f74af49249bd None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:06.279785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829f8a0-aa70-4be7-9872-f74af49249bd None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:06.281116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 703/1412] 10.222.0.55 () {54 vars in 918 bytes} [Thu Sep 8 13:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:06.285873 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a7867-14aa-4341-8078-9a5204f567db None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:06.286160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a7867-14aa-4341-8078-9a5204f567db None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:06.286455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a7867-14aa-4341-8078-9a5204f567db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:06.358066 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:06.361283 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-219a7867-14aa-4341-8078-9a5204f567db None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:06.363078 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-219a7867-14aa-4341-8078-9a5204f567db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:06.479460 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 710/1413] 10.222.0.55 () {58 vars in 1015 bytes} [Thu Sep 8 13:46:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:09.066671 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fb226-a73f-4086-91ba-caacef548ce2 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:09.067224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fb226-a73f-4086-91ba-caacef548ce2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:09.067224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fb226-a73f-4086-91ba-caacef548ce2 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:09.068562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:09.074535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294228e-2545-460f-a54a-cebb253938fd None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:09.074970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294228e-2545-460f-a54a-cebb253938fd None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:09.075079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f294228e-2545-460f-a54a-cebb253938fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:09.141973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f294228e-2545-460f-a54a-cebb253938fd None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:09.143175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f294228e-2545-460f-a54a-cebb253938fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:09.196970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:09.238986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:09.525985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fb226-a73f-4086-91ba-caacef548ce2 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:09.528032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ee3fb226-a73f-4086-91ba-caacef548ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:09.672040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fb226-a73f-4086-91ba-caacef548ce2 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:09.745517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:09.785278 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac731f-5866-4df0-a1ac-b5904f50c9fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:09.785841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac731f-5866-4df0-a1ac-b5904f50c9fa service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:09.786119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac731f-5866-4df0-a1ac-b5904f50c9fa service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:09.786385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac731f-5866-4df0-a1ac-b5904f50c9fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:09.787956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-9bac731f-5866-4df0-a1ac-b5904f50c9fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:09.876063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac731f-5866-4df0-a1ac-b5904f50c9fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:09.878055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac731f-5866-4df0-a1ac-b5904f50c9fa service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:09.896863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 705/1416] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:11.041462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2ef31-f04a-4fbc-b299-41191334bc0f None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:11.042576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2ef31-f04a-4fbc-b299-41191334bc0f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:11.042576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2ef31-f04a-4fbc-b299-41191334bc0f None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:11.043886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:46:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:11.050332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e28f0b-76b6-4da7-ab0c-4c6ceafcc423 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:11.050745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e28f0b-76b6-4da7-ab0c-4c6ceafcc423 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:11.050886 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e28f0b-76b6-4da7-ab0c-4c6ceafcc423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:11.127991 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-32e28f0b-76b6-4da7-ab0c-4c6ceafcc423 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:11.129188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-32e28f0b-76b6-4da7-ab0c-4c6ceafcc423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:11.155812 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:11.225723 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:11.376388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2ef31-f04a-4fbc-b299-41191334bc0f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:11.377932 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fba2ef31-f04a-4fbc-b299-41191334bc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:11.524254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2ef31-f04a-4fbc-b299-41191334bc0f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:11.564208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:11.635658 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30845481-6fc4-42d0-b878-d2290bb89c0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:11.636266 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30845481-6fc4-42d0-b878-d2290bb89c0d service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:11.636543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30845481-6fc4-42d0-b878-d2290bb89c0d service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:11.636813 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30845481-6fc4-42d0-b878-d2290bb89c0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:11.638442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-30845481-6fc4-42d0-b878-d2290bb89c0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:11.724945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30845481-6fc4-42d0-b878-d2290bb89c0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:11.726550 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30845481-6fc4-42d0-b878-d2290bb89c0d service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:11.744900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 713/1419] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:12.874990 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651fcba-cad0-485c-9791-e2385cbabe96 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:12.875531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651fcba-cad0-485c-9791-e2385cbabe96 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:12.875531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651fcba-cad0-485c-9791-e2385cbabe96 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:12.876870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:12.882934 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc355571-e283-497c-b828-bcefde87ac94 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:12.883393 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc355571-e283-497c-b828-bcefde87ac94 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:12.883516 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc355571-e283-497c-b828-bcefde87ac94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:12.939087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:12.962445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-bc355571-e283-497c-b828-bcefde87ac94 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:12.963649 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bc355571-e283-497c-b828-bcefde87ac94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:13.058077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:13.210814 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651fcba-cad0-485c-9791-e2385cbabe96 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:13.212271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f651fcba-cad0-485c-9791-e2385cbabe96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:13.352844 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651fcba-cad0-485c-9791-e2385cbabe96 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:13.364730 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:13.464155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f4709-ac51-4b65-9f0a-374f8eeed7b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:13.464677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f4709-ac51-4b65-9f0a-374f8eeed7b4 service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:13.464951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f4709-ac51-4b65-9f0a-374f8eeed7b4 service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:13.465218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f4709-ac51-4b65-9f0a-374f8eeed7b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:13.466830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-757f4709-ac51-4b65-9f0a-374f8eeed7b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:13.553775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f4709-ac51-4b65-9f0a-374f8eeed7b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:13.555361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757f4709-ac51-4b65-9f0a-374f8eeed7b4 service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:13.574053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 708/1422] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:14.688929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fee6f0-4827-433d-97f3-05be576601b8 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:14.689481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fee6f0-4827-433d-97f3-05be576601b8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:14.689481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fee6f0-4827-433d-97f3-05be576601b8 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:14.691084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:46:14.697046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5208c-e6d4-421b-89bd-ae30bd226b86 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:14.697460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5208c-e6d4-421b-89bd-ae30bd226b86 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:14.697610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5208c-e6d4-421b-89bd-ae30bd226b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:14.718331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:14.891277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-c4e5208c-e6d4-421b-89bd-ae30bd226b86 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:14.892468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c4e5208c-e6d4-421b-89bd-ae30bd226b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:14.986762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:46:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:15.138294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fee6f0-4827-433d-97f3-05be576601b8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:15.139811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-90fee6f0-4827-433d-97f3-05be576601b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:15.253873 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:15.288719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fee6f0-4827-433d-97f3-05be576601b8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:15.398736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39272e1-9cca-4721-a4f5-843d7478924e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:15.399362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39272e1-9cca-4721-a4f5-843d7478924e service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:15.399639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39272e1-9cca-4721-a4f5-843d7478924e service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:15.399909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39272e1-9cca-4721-a4f5-843d7478924e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:15.401502 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d39272e1-9cca-4721-a4f5-843d7478924e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:15.477686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:15.491517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39272e1-9cca-4721-a4f5-843d7478924e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:15.493093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39272e1-9cca-4721-a4f5-843d7478924e service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:15.511258 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 716/1425] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:16.612351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1714227-04c0-458e-90b8-09a42e6c27e8 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:16.612762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1714227-04c0-458e-90b8-09a42e6c27e8 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:16.612902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1714227-04c0-458e-90b8-09a42e6c27e8 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:16.614321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:16.620807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aafeb6-c400-4cdf-96d4-9301b55f87ad None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:16.621247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aafeb6-c400-4cdf-96d4-9301b55f87ad None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:16.621344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aafeb6-c400-4cdf-96d4-9301b55f87ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:16.696807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-73aafeb6-c400-4cdf-96d4-9301b55f87ad None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:16.698017 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-73aafeb6-c400-4cdf-96d4-9301b55f87ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:16.791913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:46:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:16.947161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1714227-04c0-458e-90b8-09a42e6c27e8 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:16.948681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b1714227-04c0-458e-90b8-09a42e6c27e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:17.033890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:17.095432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1714227-04c0-458e-90b8-09a42e6c27e8 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:17.211943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216d641-825e-4a73-a21b-98307b9795c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:17.212349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216d641-825e-4a73-a21b-98307b9795c9 service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:17.212547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216d641-825e-4a73-a21b-98307b9795c9 service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:17.212818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216d641-825e-4a73-a21b-98307b9795c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:17.214497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f216d641-825e-4a73-a21b-98307b9795c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:17.263805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:17.304409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216d641-825e-4a73-a21b-98307b9795c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:17.306033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216d641-825e-4a73-a21b-98307b9795c9 service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:17.324919 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 711/1428] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:18.399234 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49bc29-1da2-430e-829a-30eaafb0b1cc None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:18.399820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49bc29-1da2-430e-829a-30eaafb0b1cc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:18.399820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49bc29-1da2-430e-829a-30eaafb0b1cc None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:18.401104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:18.407883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b0817-337f-4557-837c-beefff0b725d None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:18.408290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b0817-337f-4557-837c-beefff0b725d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:18.408436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b0817-337f-4557-837c-beefff0b725d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:18.485959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-580b0817-337f-4557-837c-beefff0b725d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:18.487174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-580b0817-337f-4557-837c-beefff0b725d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:18.580993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:18.729936 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49bc29-1da2-430e-829a-30eaafb0b1cc None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:18.731412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1e49bc29-1da2-430e-829a-30eaafb0b1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:18.781019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:18.882719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49bc29-1da2-430e-829a-30eaafb0b1cc None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:19.001558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9859d-4004-487e-b2f5-8f45a48be2ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:19.002138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9859d-4004-487e-b2f5-8f45a48be2ef service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:19.002417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9859d-4004-487e-b2f5-8f45a48be2ef service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:19.002690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9859d-4004-487e-b2f5-8f45a48be2ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:19.004285 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-65a9859d-4004-487e-b2f5-8f45a48be2ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:19.011979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:19.094959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9859d-4004-487e-b2f5-8f45a48be2ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:19.096572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9859d-4004-487e-b2f5-8f45a48be2ef service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:19.114744 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 719/1431] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:20.158572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7becd-3128-4049-a7c9-64baf4d6fdc1 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:20.159190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7becd-3128-4049-a7c9-64baf4d6fdc1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:20.159190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7becd-3128-4049-a7c9-64baf4d6fdc1 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:20.160488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:20.166860 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f58fa-5583-4dc8-8fc9-0bcbe10f45a7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:20.167347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f58fa-5583-4dc8-8fc9-0bcbe10f45a7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:20.168003 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f58fa-5583-4dc8-8fc9-0bcbe10f45a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:20.243015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-630f58fa-5583-4dc8-8fc9-0bcbe10f45a7 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:20.244196 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-630f58fa-5583-4dc8-8fc9-0bcbe10f45a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:20.309892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:20.339902 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:20.488581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7becd-3128-4049-a7c9-64baf4d6fdc1 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:20.490049 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-fad7becd-3128-4049-a7c9-64baf4d6fdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:20.507494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:20.633781 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7becd-3128-4049-a7c9-64baf4d6fdc1 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:20.718275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:20.744993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5deef53-f61d-42f2-a57a-365eaf82a7d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:20.746076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5deef53-f61d-42f2-a57a-365eaf82a7d2 service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:20.746352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5deef53-f61d-42f2-a57a-365eaf82a7d2 service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:20.746612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5deef53-f61d-42f2-a57a-365eaf82a7d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:20.748182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d5deef53-f61d-42f2-a57a-365eaf82a7d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:20.833846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5deef53-f61d-42f2-a57a-365eaf82a7d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:20.835411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5deef53-f61d-42f2-a57a-365eaf82a7d2 service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:20.853069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 714/1434] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:21.903690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee87495-ad9d-484b-8e28-6ec80ead4263 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:21.904271 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee87495-ad9d-484b-8e28-6ec80ead4263 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:21.904271 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee87495-ad9d-484b-8e28-6ec80ead4263 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:21.905605 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:21.912019 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0142dd1-b4fe-4bd1-be0b-307bccf30eb1 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:21.912468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0142dd1-b4fe-4bd1-be0b-307bccf30eb1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:21.912574 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0142dd1-b4fe-4bd1-be0b-307bccf30eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:21.989503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-f0142dd1-b4fe-4bd1-be0b-307bccf30eb1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:21.990746 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f0142dd1-b4fe-4bd1-be0b-307bccf30eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:22.032428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:22.090195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:46:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:22.244787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee87495-ad9d-484b-8e28-6ec80ead4263 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:22.246281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-aee87495-ad9d-484b-8e28-6ec80ead4263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:22.392078 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee87495-ad9d-484b-8e28-6ec80ead4263 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:22.445069 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:22.506450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf224a-6db4-41ad-8572-3681bdcfd767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:22.506978 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf224a-6db4-41ad-8572-3681bdcfd767 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:22.507259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf224a-6db4-41ad-8572-3681bdcfd767 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:22.507546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf224a-6db4-41ad-8572-3681bdcfd767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:22.509129 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-02bf224a-6db4-41ad-8572-3681bdcfd767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:22.597082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf224a-6db4-41ad-8572-3681bdcfd767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:22.599179 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf224a-6db4-41ad-8572-3681bdcfd767 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:22.616343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 722/1437] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:23.669992 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b5831-d033-4941-9153-fbefe0a58953 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:23.670564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b5831-d033-4941-9153-fbefe0a58953 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:23.670564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b5831-d033-4941-9153-fbefe0a58953 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:23.671893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:23.678036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9abb4-9d28-4e7e-82ac-6715ba3cd22c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:23.678475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9abb4-9d28-4e7e-82ac-6715ba3cd22c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:23.678581 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d9abb4-9d28-4e7e-82ac-6715ba3cd22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:23.753211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-55d9abb4-9d28-4e7e-82ac-6715ba3cd22c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:23.754445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-55d9abb4-9d28-4e7e-82ac-6715ba3cd22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:23.756496 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:23.851526 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:24.000908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b5831-d033-4941-9153-fbefe0a58953 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:24.002407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d50b5831-d033-4941-9153-fbefe0a58953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:24.147469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b5831-d033-4941-9153-fbefe0a58953 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:24.178801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:24.269959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd69a3-b324-46fe-85cd-669971b96525 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:24.270476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd69a3-b324-46fe-85cd-669971b96525 service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:24.270748 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd69a3-b324-46fe-85cd-669971b96525 service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:24.271040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd69a3-b324-46fe-85cd-669971b96525 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:24.272627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e8cd69a3-b324-46fe-85cd-669971b96525 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:24.362338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd69a3-b324-46fe-85cd-669971b96525 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:24.363918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd69a3-b324-46fe-85cd-669971b96525 service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:24.381991 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 717/1440] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:26.439126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82b00d-770f-48e7-9a52-d37cceb86bbf None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:26.439657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82b00d-770f-48e7-9a52-d37cceb86bbf None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:26.439788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be82b00d-770f-48e7-9a52-d37cceb86bbf None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:26.441074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:26.446572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b4988-2c94-45a3-9128-699d57182712 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:26.446973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b4988-2c94-45a3-9128-699d57182712 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:26.447127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b4988-2c94-45a3-9128-699d57182712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:26.488221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:26.513590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-974b4988-2c94-45a3-9128-699d57182712 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:26.514825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-974b4988-2c94-45a3-9128-699d57182712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:26.610078 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:26.891640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82b00d-770f-48e7-9a52-d37cceb86bbf None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:26.893178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-be82b00d-770f-48e7-9a52-d37cceb86bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:26.997426 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be82b00d-770f-48e7-9a52-d37cceb86bbf None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:27.030208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:27.071506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb371f-ef03-4706-be66-53fe7ce58086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:27.072109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb371f-ef03-4706-be66-53fe7ce58086 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:27.072390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb371f-ef03-4706-be66-53fe7ce58086 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:27.072663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb371f-ef03-4706-be66-53fe7ce58086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:27.074348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-42cb371f-ef03-4706-be66-53fe7ce58086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:27.180794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb371f-ef03-4706-be66-53fe7ce58086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:27.182184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb371f-ef03-4706-be66-53fe7ce58086 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:27.200973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 725/1443] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:29.578271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08372588-53da-401c-b482-55e9aaece006 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:29.578849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08372588-53da-401c-b482-55e9aaece006 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:29.578849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08372588-53da-401c-b482-55e9aaece006 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:29.580195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:29.585542 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580365b-4045-4086-b823-9c5635a0edeb None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:29.585964 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580365b-4045-4086-b823-9c5635a0edeb None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:29.586113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580365b-4045-4086-b823-9c5635a0edeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:29.652217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0580365b-4045-4086-b823-9c5635a0edeb None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:29.653961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0580365b-4045-4086-b823-9c5635a0edeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:29.666535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:29.750516 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:30.037012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08372588-53da-401c-b482-55e9aaece006 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:30.038588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-08372588-53da-401c-b482-55e9aaece006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:30.141788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08372588-53da-401c-b482-55e9aaece006 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:30.154405 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:30.215717 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc049bd4-bc18-418e-a984-c4c9f2f67c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:30.216241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc049bd4-bc18-418e-a984-c4c9f2f67c2f service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:30.216519 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc049bd4-bc18-418e-a984-c4c9f2f67c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:30.216791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc049bd4-bc18-418e-a984-c4c9f2f67c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:30.218416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-dc049bd4-bc18-418e-a984-c4c9f2f67c2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:30.330762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc049bd4-bc18-418e-a984-c4c9f2f67c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:30.332188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc049bd4-bc18-418e-a984-c4c9f2f67c2f service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:30.350312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 720/1446] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:32.799860 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c68f2c-b905-4ab0-8c44-cd0aad7d5885 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:32.800391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c68f2c-b905-4ab0-8c44-cd0aad7d5885 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:32.800391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c68f2c-b905-4ab0-8c44-cd0aad7d5885 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:32.801784 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:32.807113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f661ee-912f-403c-a707-26348ff99ad9 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:32.807579 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f661ee-912f-403c-a707-26348ff99ad9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:32.807678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f661ee-912f-403c-a707-26348ff99ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:32.855444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:32.874342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-24f661ee-912f-403c-a707-26348ff99ad9 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:32.875527 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-24f661ee-912f-403c-a707-26348ff99ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:32.977476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:46:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:33.263508 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c68f2c-b905-4ab0-8c44-cd0aad7d5885 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:33.265064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-28c68f2c-b905-4ab0-8c44-cd0aad7d5885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:33.368151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c68f2c-b905-4ab0-8c44-cd0aad7d5885 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:33.411098 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:33.446184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c90dd-f2b5-4b7b-ba13-93ce79ed526a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:33.446790 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c90dd-f2b5-4b7b-ba13-93ce79ed526a service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:33.447066 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c90dd-f2b5-4b7b-ba13-93ce79ed526a service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:33.447336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c90dd-f2b5-4b7b-ba13-93ce79ed526a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:33.448916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-824c90dd-f2b5-4b7b-ba13-93ce79ed526a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:33.554604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c90dd-f2b5-4b7b-ba13-93ce79ed526a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:33.555962 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c90dd-f2b5-4b7b-ba13-93ce79ed526a service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:33.574553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 728/1449] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:35.836710 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3b5b2-bd85-490c-9045-419ac57215b7 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:35.837331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3b5b2-bd85-490c-9045-419ac57215b7 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:35.837331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3b5b2-bd85-490c-9045-419ac57215b7 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:35.838610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:35.844406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d72f8-0fd2-451c-a057-591b2a8e76d3 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:35.844803 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d72f8-0fd2-451c-a057-591b2a8e76d3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:35.844969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d72f8-0fd2-451c-a057-591b2a8e76d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:35.911975 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-bb8d72f8-0fd2-451c-a057-591b2a8e76d3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:35.913153 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bb8d72f8-0fd2-451c-a057-591b2a8e76d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:35.933275 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:36.009749 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:36.298387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3b5b2-bd85-490c-9045-419ac57215b7 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:36.299924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-95e3b5b2-bd85-490c-9045-419ac57215b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:36.403434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3b5b2-bd85-490c-9045-419ac57215b7 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:36.426339 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:36.481528 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188d37d-34f0-4ed4-b12c-e97ba4b953bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:36.482476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188d37d-34f0-4ed4-b12c-e97ba4b953bc service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:36.482765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188d37d-34f0-4ed4-b12c-e97ba4b953bc service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:36.483035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188d37d-34f0-4ed4-b12c-e97ba4b953bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:36.484742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4188d37d-34f0-4ed4-b12c-e97ba4b953bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:36.599290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188d37d-34f0-4ed4-b12c-e97ba4b953bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:36.600672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188d37d-34f0-4ed4-b12c-e97ba4b953bc service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:36.618226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 723/1452] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:38.819602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae2ea0-8679-4f46-8426-a95052e994f7 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:38.820194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae2ea0-8679-4f46-8426-a95052e994f7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:38.820194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae2ea0-8679-4f46-8426-a95052e994f7 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:38.821574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:38.827471 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafc9f3-b23f-4188-9c31-7629ec8e35a5 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:38.827877 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafc9f3-b23f-4188-9c31-7629ec8e35a5 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:38.828021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cafc9f3-b23f-4188-9c31-7629ec8e35a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:38.891463 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:38.894693 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-2cafc9f3-b23f-4188-9c31-7629ec8e35a5 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:38.896460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2cafc9f3-b23f-4188-9c31-7629ec8e35a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:38.999391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:39.288402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae2ea0-8679-4f46-8426-a95052e994f7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:39.289959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-48ae2ea0-8679-4f46-8426-a95052e994f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:39.395281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae2ea0-8679-4f46-8426-a95052e994f7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:39.445029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:39.470079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5efd3-47ff-4618-a190-4f27f39a972f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:39.470687 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5efd3-47ff-4618-a190-4f27f39a972f service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:39.470959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5efd3-47ff-4618-a190-4f27f39a972f service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:39.471223 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5efd3-47ff-4618-a190-4f27f39a972f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:39.472794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ace5efd3-47ff-4618-a190-4f27f39a972f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:39.578144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5efd3-47ff-4618-a190-4f27f39a972f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:39.579493 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace5efd3-47ff-4618-a190-4f27f39a972f service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:39.598072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 731/1455] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:41.819140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ebece-cafd-4761-86de-b73a1cc98961 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:41.819756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ebece-cafd-4761-86de-b73a1cc98961 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:41.819756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ebece-cafd-4761-86de-b73a1cc98961 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:41.821136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:41.826986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed695c3-8b7e-4dfb-9e3f-01e84af22dbf None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:41.827424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed695c3-8b7e-4dfb-9e3f-01e84af22dbf None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:41.827520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed695c3-8b7e-4dfb-9e3f-01e84af22dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:41.893655 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-aed695c3-8b7e-4dfb-9e3f-01e84af22dbf None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:41.894889 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-aed695c3-8b7e-4dfb-9e3f-01e84af22dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:41.924894 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:42.002081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:46:42.290559 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ebece-cafd-4761-86de-b73a1cc98961 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:42.292104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-6c4ebece-cafd-4761-86de-b73a1cc98961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:42.400500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ebece-cafd-4761-86de-b73a1cc98961 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:42.429002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:42.479241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded49f4-e427-41d1-9a3d-220c836e1aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:42.479842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded49f4-e427-41d1-9a3d-220c836e1aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:42.480121 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded49f4-e427-41d1-9a3d-220c836e1aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:42.480393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded49f4-e427-41d1-9a3d-220c836e1aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:42.482624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4ded49f4-e427-41d1-9a3d-220c836e1aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:42.591250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded49f4-e427-41d1-9a3d-220c836e1aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:42.592588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded49f4-e427-41d1-9a3d-220c836e1aa0 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:42.610510 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 726/1458] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:44.813341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6409e5b-361f-415b-ae9e-33d7f81f972a None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:44.813902 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6409e5b-361f-415b-ae9e-33d7f81f972a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:44.813902 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6409e5b-361f-415b-ae9e-33d7f81f972a None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:44.815254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:44.820733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65018a5c-a834-4420-9244-de953d48eea4 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:44.821176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65018a5c-a834-4420-9244-de953d48eea4 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:44.821275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65018a5c-a834-4420-9244-de953d48eea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:44.887225 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-65018a5c-a834-4420-9244-de953d48eea4 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:44.888356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-65018a5c-a834-4420-9244-de953d48eea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:44.898132 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:44.986786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:45.269640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6409e5b-361f-415b-ae9e-33d7f81f972a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:45.271398 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b6409e5b-361f-415b-ae9e-33d7f81f972a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:45.373920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6409e5b-361f-415b-ae9e-33d7f81f972a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:45.430607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:45.449270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cd3b0-9c14-456c-9bc9-f6b5fedd856f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:45.449908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cd3b0-9c14-456c-9bc9-f6b5fedd856f service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:45.450226 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cd3b0-9c14-456c-9bc9-f6b5fedd856f service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:45.450520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cd3b0-9c14-456c-9bc9-f6b5fedd856f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:45.452068 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c72cd3b0-9c14-456c-9bc9-f6b5fedd856f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:45.559505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cd3b0-9c14-456c-9bc9-f6b5fedd856f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:45.561354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cd3b0-9c14-456c-9bc9-f6b5fedd856f service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:45.579191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 734/1461] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:47.790866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49813e81-d803-401c-ac5a-29605ca8d9e1 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:47.791409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49813e81-d803-401c-ac5a-29605ca8d9e1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:47.791409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49813e81-d803-401c-ac5a-29605ca8d9e1 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:47.792673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:47.798574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31feaed1-dce0-486d-9375-78ad8d966951 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:47.798988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31feaed1-dce0-486d-9375-78ad8d966951 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:47.799145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31feaed1-dce0-486d-9375-78ad8d966951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:47.865575 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-31feaed1-dce0-486d-9375-78ad8d966951 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:47.866804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-31feaed1-dce0-486d-9375-78ad8d966951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:47.905338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:47.967764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:48.251267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49813e81-d803-401c-ac5a-29605ca8d9e1 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:48.252804 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-49813e81-d803-401c-ac5a-29605ca8d9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:48.355892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49813e81-d803-401c-ac5a-29605ca8d9e1 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:48.397308 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:48.433800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c117e-e317-4a50-8c6f-3a6e0dd716ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:48.434400 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c117e-e317-4a50-8c6f-3a6e0dd716ef service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:48.434670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c117e-e317-4a50-8c6f-3a6e0dd716ef service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:48.434933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c117e-e317-4a50-8c6f-3a6e0dd716ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:48.436522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-647c117e-e317-4a50-8c6f-3a6e0dd716ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:48.546420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c117e-e317-4a50-8c6f-3a6e0dd716ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:48.547838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c117e-e317-4a50-8c6f-3a6e0dd716ef service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:48.565817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 729/1464] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:50.762642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053e41e-38b3-4553-9c2d-7d73a20cbc29 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:50.763712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053e41e-38b3-4553-9c2d-7d73a20cbc29 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:50.763712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053e41e-38b3-4553-9c2d-7d73a20cbc29 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:50.765035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:46:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:50.770631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e52db-e969-4140-b443-229e46d36b27 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:50.771061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e52db-e969-4140-b443-229e46d36b27 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:50.771204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e52db-e969-4140-b443-229e46d36b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:50.838196 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-cf5e52db-e969-4140-b443-229e46d36b27 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:50.839417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-cf5e52db-e969-4140-b443-229e46d36b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:50.858066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:50.944245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:51.237477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053e41e-38b3-4553-9c2d-7d73a20cbc29 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:51.239097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9053e41e-38b3-4553-9c2d-7d73a20cbc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:51.343136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053e41e-38b3-4553-9c2d-7d73a20cbc29 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:51.411193 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:51.418438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0fc94-6f31-467d-bd60-b05210608e44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:51.419074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0fc94-6f31-467d-bd60-b05210608e44 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:51.419955 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0fc94-6f31-467d-bd60-b05210608e44 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:51.420226 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0fc94-6f31-467d-bd60-b05210608e44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:51.421871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b5e0fc94-6f31-467d-bd60-b05210608e44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:51.533034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0fc94-6f31-467d-bd60-b05210608e44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:51.534482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0fc94-6f31-467d-bd60-b05210608e44 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:51.553836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 737/1467] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:46:53.713978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d6201-776c-4a71-982e-a5d683af3589 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:53.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d6201-776c-4a71-982e-a5d683af3589 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:53.714517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d6201-776c-4a71-982e-a5d683af3589 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:53.716180 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:46:53.721266 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390aa94-7e49-480b-85af-66d825820423 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:53.721697 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390aa94-7e49-480b-85af-66d825820423 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:53.721986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390aa94-7e49-480b-85af-66d825820423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:53.787104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f390aa94-7e49-480b-85af-66d825820423 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:46:53.788300 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f390aa94-7e49-480b-85af-66d825820423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:53.836127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:53.883877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:46:54.167898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d6201-776c-4a71-982e-a5d683af3589 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:46:54.170105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-6b3d6201-776c-4a71-982e-a5d683af3589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:54.271985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d6201-776c-4a71-982e-a5d683af3589 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:46:54.319343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:54.345453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79fdf7c-6377-4b6f-8aff-fb8bfc187cd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:46:54.346084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fdf7c-6377-4b6f-8aff-fb8bfc187cd5 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:46:54.346361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fdf7c-6377-4b6f-8aff-fb8bfc187cd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:46:54.346633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79fdf7c-6377-4b6f-8aff-fb8bfc187cd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:46:54.348209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f79fdf7c-6377-4b6f-8aff-fb8bfc187cd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:46:54.455132 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79fdf7c-6377-4b6f-8aff-fb8bfc187cd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:46:54.456489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79fdf7c-6377-4b6f-8aff-fb8bfc187cd5 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:46:54.474870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 732/1470] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:03.882352 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=22 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:04.472554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=66 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:14.516776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8963c3-7d19-4728-a928-94309a53ae3d None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:14.517300 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8963c3-7d19-4728-a928-94309a53ae3d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:14.517415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8963c3-7d19-4728-a928-94309a53ae3d None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:14.518759 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 739/1471] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:14.523942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde1aed-53ee-436e-851e-e05ab90cafbe None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:14.524343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde1aed-53ee-436e-851e-e05ab90cafbe None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:14.524494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde1aed-53ee-436e-851e-e05ab90cafbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:14.590750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-edde1aed-53ee-436e-851e-e05ab90cafbe None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:14.591943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-edde1aed-53ee-436e-851e-e05ab90cafbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:14.686607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 733/1472] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:14.990182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08230239-21b9-407e-94f5-7081c8b6a9d3 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:14.990471 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08230239-21b9-407e-94f5-7081c8b6a9d3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:14.990742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08230239-21b9-407e-94f5-7081c8b6a9d3 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:14.992104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 740/1473] 10.222.0.55 () {54 vars in 961 bytes} [Thu Sep 8 13:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:14.997268 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d247e4-c2dd-4cfc-91cc-b78400f046b1 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:14.997553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d247e4-c2dd-4cfc-91cc-b78400f046b1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:14.997923 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d247e4-c2dd-4cfc-91cc-b78400f046b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:15.073006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-e9d247e4-c2dd-4cfc-91cc-b78400f046b1 None None] MFA Rules not processed for user `4e27f5b81c704dbebbd6a83b2713e237`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:15.074235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e9d247e4-c2dd-4cfc-91cc-b78400f046b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:15.094792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:15.149367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 734/1474] 10.222.0.55 () {58 vars in 1058 bytes} [Thu Sep 8 13:47:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:15.157918 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08230239-21b9-407e-94f5-7081c8b6a9d3 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:15.159490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-08230239-21b9-407e-94f5-7081c8b6a9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:15.266841 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08230239-21b9-407e-94f5-7081c8b6a9d3 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:15.340674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2bce8-a2cb-4e56-a118-22acbf7ff85f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e27f5b81c704dbebbd6a83b2713e237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:15.341276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2bce8-a2cb-4e56-a118-22acbf7ff85f service glance] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:15.341555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2bce8-a2cb-4e56-a118-22acbf7ff85f service glance] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:15.341858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2bce8-a2cb-4e56-a118-22acbf7ff85f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:15.344088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b2a2bce8-a2cb-4e56-a118-22acbf7ff85f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:15.399865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:15.455322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2bce8-a2cb-4e56-a118-22acbf7ff85f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:15.456719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2bce8-a2cb-4e56-a118-22acbf7ff85f service glance] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:15.475127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 741/1475] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:17.708714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dada3-9abf-4105-b0e1-58ba1b50a765 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:17.709240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dada3-9abf-4105-b0e1-58ba1b50a765 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:17.709240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87dada3-9abf-4105-b0e1-58ba1b50a765 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:17.710558 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 735/1476] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:17.715767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651381c-c346-4a82-88b4-d443fbec06c4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:17.716204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651381c-c346-4a82-88b4-d443fbec06c4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:17.716305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651381c-c346-4a82-88b4-d443fbec06c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:17.781155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9651381c-c346-4a82-88b4-d443fbec06c4 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:17.782858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9651381c-c346-4a82-88b4-d443fbec06c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:17.877402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 742/1477] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:18.149632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87dada3-9abf-4105-b0e1-58ba1b50a765 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:18.151217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a87dada3-9abf-4105-b0e1-58ba1b50a765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:18.275601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87dada3-9abf-4105-b0e1-58ba1b50a765 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:18.325071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:18.369462 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb758a-b696-4577-b57a-44cb15f11afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:18.370107 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb758a-b696-4577-b57a-44cb15f11afa admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:18.370384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb758a-b696-4577-b57a-44cb15f11afa admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:18.370656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb758a-b696-4577-b57a-44cb15f11afa admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:18.371292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb758a-b696-4577-b57a-44cb15f11afa admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:18.373022 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: WARNING py.warnings [None req-1bcb758a-b696-4577-b57a-44cb15f11afa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:47:18.373022 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: warnings.warn(msg) Sep 08 13:47:18.373022 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: Sep 08 13:47:18.374778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb758a-b696-4577-b57a-44cb15f11afa admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:18.394956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 736/1478] 10.222.0.55 () {56 vars in 1108 bytes} [Thu Sep 8 13:47:18 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:47:20.569642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db364a44-407e-4d3b-9a30-8df754078578 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:20.570276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db364a44-407e-4d3b-9a30-8df754078578 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:20.570276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db364a44-407e-4d3b-9a30-8df754078578 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:20.571854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 743/1479] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:20.577034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e2e13-1614-4fcc-af09-3dd4ce3a5269 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:20.577482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e2e13-1614-4fcc-af09-3dd4ce3a5269 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:20.577589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e2e13-1614-4fcc-af09-3dd4ce3a5269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:20.643688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-379e2e13-1614-4fcc-af09-3dd4ce3a5269 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:20.644921 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-379e2e13-1614-4fcc-af09-3dd4ce3a5269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:20.743311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 737/1480] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:21.042624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db364a44-407e-4d3b-9a30-8df754078578 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:21.044208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-db364a44-407e-4d3b-9a30-8df754078578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:21.054562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:21.187340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db364a44-407e-4d3b-9a30-8df754078578 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:21.267228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:21.303380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18700eb7-3867-4ff3-a504-994d8e83f196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:21.304403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18700eb7-3867-4ff3-a504-994d8e83f196 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:21.304680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18700eb7-3867-4ff3-a504-994d8e83f196 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:21.304950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18700eb7-3867-4ff3-a504-994d8e83f196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:21.306570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-18700eb7-3867-4ff3-a504-994d8e83f196 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:21.396985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18700eb7-3867-4ff3-a504-994d8e83f196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:21.398674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18700eb7-3867-4ff3-a504-994d8e83f196 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:21.416016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 744/1481] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:23.743199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5573d-1ea1-435e-a491-d0d4442a9883 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:23.743784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5573d-1ea1-435e-a491-d0d4442a9883 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:23.743784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5573d-1ea1-435e-a491-d0d4442a9883 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:23.745097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 738/1482] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:23.750398 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b7026-2851-4e23-8985-848816fc2c76 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:23.750839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b7026-2851-4e23-8985-848816fc2c76 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:23.750937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b7026-2851-4e23-8985-848816fc2c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:23.816298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d28b7026-2851-4e23-8985-848816fc2c76 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:23.817497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d28b7026-2851-4e23-8985-848816fc2c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:23.863111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:23.914117 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 745/1483] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:24.205941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5573d-1ea1-435e-a491-d0d4442a9883 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:24.207550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-24c5573d-1ea1-435e-a491-d0d4442a9883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:24.231105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:24.311936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5573d-1ea1-435e-a491-d0d4442a9883 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:24.385620 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71f482-cea4-41d3-9da3-a4383ada1c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:24.386253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71f482-cea4-41d3-9da3-a4383ada1c6b service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:24.386531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71f482-cea4-41d3-9da3-a4383ada1c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:24.386798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71f482-cea4-41d3-9da3-a4383ada1c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:24.388409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-cc71f482-cea4-41d3-9da3-a4383ada1c6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:24.462045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:24.498129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71f482-cea4-41d3-9da3-a4383ada1c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:24.499589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71f482-cea4-41d3-9da3-a4383ada1c6b service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:24.519419 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 739/1484] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:26.695472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b86b3-d033-40e4-a9a6-b882a1fd47e7 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:26.696012 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b86b3-d033-40e4-a9a6-b882a1fd47e7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:26.696012 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b86b3-d033-40e4-a9a6-b882a1fd47e7 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:26.697598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 746/1485] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:26.703305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d081a7b-f7b6-4682-91bd-e73967da61eb None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:26.703693 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d081a7b-f7b6-4682-91bd-e73967da61eb None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:26.703853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d081a7b-f7b6-4682-91bd-e73967da61eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:26.768662 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-6d081a7b-f7b6-4682-91bd-e73967da61eb None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:26.769896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-6d081a7b-f7b6-4682-91bd-e73967da61eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:26.870231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 740/1486] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:27.162440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b86b3-d033-40e4-a9a6-b882a1fd47e7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:27.163908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8d3b86b3-d033-40e4-a9a6-b882a1fd47e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:27.265655 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b86b3-d033-40e4-a9a6-b882a1fd47e7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:27.336109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:27.337700 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ba168-e84b-4552-bda0-2d6ad52810d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:27.338377 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ba168-e84b-4552-bda0-2d6ad52810d9 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:27.338842 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ba168-e84b-4552-bda0-2d6ad52810d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:27.339325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ba168-e84b-4552-bda0-2d6ad52810d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:27.341155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a83ba168-e84b-4552-bda0-2d6ad52810d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:27.567388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ba168-e84b-4552-bda0-2d6ad52810d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:27.568735 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83ba168-e84b-4552-bda0-2d6ad52810d9 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:27.586688 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 747/1487] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:29.834196 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c5149-2adb-4117-948b-4eb2db6645cb None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:29.834767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c5149-2adb-4117-948b-4eb2db6645cb None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:29.834767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c5149-2adb-4117-948b-4eb2db6645cb None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:29.836087 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 741/1488] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:47:29.841301 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69a4e5-98f5-4e56-b7a6-83aae68cf24e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:29.841760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69a4e5-98f5-4e56-b7a6-83aae68cf24e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:29.841887 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69a4e5-98f5-4e56-b7a6-83aae68cf24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:29.909066 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8d69a4e5-98f5-4e56-b7a6-83aae68cf24e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:29.910297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8d69a4e5-98f5-4e56-b7a6-83aae68cf24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:29.962826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:30.006404 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 748/1489] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:47:30.295306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c5149-2adb-4117-948b-4eb2db6645cb None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:30.297372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b43c5149-2adb-4117-948b-4eb2db6645cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:30.333685 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:30.400267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c5149-2adb-4117-948b-4eb2db6645cb None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:30.473567 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58134c-568b-4135-8c3e-ef0d2e1d29a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:30.474233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58134c-568b-4135-8c3e-ef0d2e1d29a6 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:30.474509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58134c-568b-4135-8c3e-ef0d2e1d29a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:30.474779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58134c-568b-4135-8c3e-ef0d2e1d29a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:30.476367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ed58134c-568b-4135-8c3e-ef0d2e1d29a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:30.557539 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:30.587923 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58134c-568b-4135-8c3e-ef0d2e1d29a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:30.589361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58134c-568b-4135-8c3e-ef0d2e1d29a6 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:30.607728 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 742/1490] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:32.958045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bc7ee-127e-409d-a9a0-803cfb9c2b23 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:32.958566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bc7ee-127e-409d-a9a0-803cfb9c2b23 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:32.958673 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4bc7ee-127e-409d-a9a0-803cfb9c2b23 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:32.959961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 749/1491] 10.222.0.55 () {54 vars in 927 bytes} [Thu Sep 8 13:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:47:32.965105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596ee31-d109-4f08-802f-91c768123ed5 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:32.965530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596ee31-d109-4f08-802f-91c768123ed5 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:32.965688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596ee31-d109-4f08-802f-91c768123ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:33.032402 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-8596ee31-d109-4f08-802f-91c768123ed5 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:47:33.033633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8596ee31-d109-4f08-802f-91c768123ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:33.129004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 743/1492] 10.222.0.55 () {58 vars in 1024 bytes} [Thu Sep 8 13:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:47:33.430220 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4bc7ee-127e-409d-a9a0-803cfb9c2b23 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:47:33.432766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2c4bc7ee-127e-409d-a9a0-803cfb9c2b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:33.537292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4bc7ee-127e-409d-a9a0-803cfb9c2b23 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:47:33.610309 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c7f11-734e-4b4b-9222-16c94e21a8d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:47:33.610911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c7f11-734e-4b4b-9222-16c94e21a8d4 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:47:33.611189 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c7f11-734e-4b4b-9222-16c94e21a8d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:47:33.611458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c7f11-734e-4b4b-9222-16c94e21a8d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:47:33.613047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3b6c7f11-734e-4b4b-9222-16c94e21a8d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:47:33.623059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:33.722323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c7f11-734e-4b4b-9222-16c94e21a8d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:47:33.723656 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c7f11-734e-4b4b-9222-16c94e21a8d4 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:47:33.741117 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 750/1493] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:47:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:43.127464 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=82 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:43.738804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=50 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:46.284921 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f70261-a765-4717-87eb-f8084a64e2a1 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:46.285485 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f70261-a765-4717-87eb-f8084a64e2a1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:46.285485 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f70261-a765-4717-87eb-f8084a64e2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:46.365787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-89f70261-a765-4717-87eb-f8084a64e2a1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:48:46.367016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-89f70261-a765-4717-87eb-f8084a64e2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:46.466965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 744/1494] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:48:46.475905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4ebe0-fff1-490a-b80b-ee40bd72995c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:46.476435 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4ebe0-fff1-490a-b80b-ee40bd72995c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:46.476435 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4ebe0-fff1-490a-b80b-ee40bd72995c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:46.550741 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-85b4ebe0-fff1-490a-b80b-ee40bd72995c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:48:46.551863 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-85b4ebe0-fff1-490a-b80b-ee40bd72995c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:46.646010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 751/1495] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:48:46.650672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f70261-a765-4717-87eb-f8084a64e2a1 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:46.651937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-89f70261-a765-4717-87eb-f8084a64e2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:46.664242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f70261-a765-4717-87eb-f8084a64e2a1 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:46.727835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:46.762368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04942aa7-bd06-4d65-bfc8-b988662fd003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:46.762976 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04942aa7-bd06-4d65-bfc8-b988662fd003 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:46.763249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04942aa7-bd06-4d65-bfc8-b988662fd003 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:46.763510 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04942aa7-bd06-4d65-bfc8-b988662fd003 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:46.764146 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04942aa7-bd06-4d65-bfc8-b988662fd003 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:48:46.764440 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04942aa7-bd06-4d65-bfc8-b988662fd003 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:46.765437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: WARNING py.warnings [None req-04942aa7-bd06-4d65-bfc8-b988662fd003 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:48:46.765437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: warnings.warn(msg) Sep 08 13:48:46.765437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: Sep 08 13:48:46.766344 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04942aa7-bd06-4d65-bfc8-b988662fd003 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:46.777459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 745/1496] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:48:46.785995 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4ebe0-fff1-490a-b80b-ee40bd72995c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:46.795307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b4ebe0-fff1-490a-b80b-ee40bd72995c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:46.857823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:46.890288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb43ee8-a12c-4566-b4a5-bf23ba4f77d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:46.890883 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb43ee8-a12c-4566-b4a5-bf23ba4f77d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:46.891154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb43ee8-a12c-4566-b4a5-bf23ba4f77d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:46.891417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb43ee8-a12c-4566-b4a5-bf23ba4f77d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:46.892040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb43ee8-a12c-4566-b4a5-bf23ba4f77d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:46.894270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-7eb43ee8-a12c-4566-b4a5-bf23ba4f77d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:48:46.894270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Sep 08 13:48:46.894270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: Sep 08 13:48:46.895453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb43ee8-a12c-4566-b4a5-bf23ba4f77d7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:46.951227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 752/1497] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:48:46 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:48:46.956337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04942aa7-bd06-4d65-bfc8-b988662fd003 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:46.958035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-04942aa7-bd06-4d65-bfc8-b988662fd003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:46.992236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04942aa7-bd06-4d65-bfc8-b988662fd003 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:47.087983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16dfe03-f1a3-4576-a5dc-622719742310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:47.088609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16dfe03-f1a3-4576-a5dc-622719742310 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:47.088889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16dfe03-f1a3-4576-a5dc-622719742310 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:47.089157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16dfe03-f1a3-4576-a5dc-622719742310 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:47.089821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16dfe03-f1a3-4576-a5dc-622719742310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:47.090845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: WARNING py.warnings [None req-e16dfe03-f1a3-4576-a5dc-622719742310 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:48:47.090845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: warnings.warn(msg) Sep 08 13:48:47.090845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: Sep 08 13:48:47.092083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16dfe03-f1a3-4576-a5dc-622719742310 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:47.134451 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 746/1498] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:48:46 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:48:47.138583 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb43ee8-a12c-4566-b4a5-bf23ba4f77d7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:47.140140 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7eb43ee8-a12c-4566-b4a5-bf23ba4f77d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:47.202574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb43ee8-a12c-4566-b4a5-bf23ba4f77d7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:47.285190 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:47.296579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efd16c-5f0e-4235-bfd3-7fa33b196607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:47.297186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efd16c-5f0e-4235-bfd3-7fa33b196607 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:47.297456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efd16c-5f0e-4235-bfd3-7fa33b196607 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:47.297718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efd16c-5f0e-4235-bfd3-7fa33b196607 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:47.298382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efd16c-5f0e-4235-bfd3-7fa33b196607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:47.299403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-f2efd16c-5f0e-4235-bfd3-7fa33b196607 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:48:47.299403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Sep 08 13:48:47.299403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: Sep 08 13:48:47.300281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efd16c-5f0e-4235-bfd3-7fa33b196607 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:47.312475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 753/1499] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:48:47.317148 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16dfe03-f1a3-4576-a5dc-622719742310 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:47.318855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e16dfe03-f1a3-4576-a5dc-622719742310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:47.331196 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16dfe03-f1a3-4576-a5dc-622719742310 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:47.356386 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:47.433456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f61f65-c5d7-4a22-b89c-93d8c88c1027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:47.434098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f61f65-c5d7-4a22-b89c-93d8c88c1027 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:47.434375 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f61f65-c5d7-4a22-b89c-93d8c88c1027 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:47.434644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f61f65-c5d7-4a22-b89c-93d8c88c1027 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:47.435283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f61f65-c5d7-4a22-b89c-93d8c88c1027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:47.436311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: WARNING py.warnings [None req-79f61f65-c5d7-4a22-b89c-93d8c88c1027 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:48:47.436311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: warnings.warn(msg) Sep 08 13:48:47.436311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: Sep 08 13:48:47.437176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f61f65-c5d7-4a22-b89c-93d8c88c1027 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:47.450029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 747/1500] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:48:47.454532 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efd16c-5f0e-4235-bfd3-7fa33b196607 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:47.456694 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f2efd16c-5f0e-4235-bfd3-7fa33b196607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:47.491049 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efd16c-5f0e-4235-bfd3-7fa33b196607 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:47.585913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c284f0d-2127-44e4-a892-8528813e83bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:47.586562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c284f0d-2127-44e4-a892-8528813e83bf admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:47.586832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c284f0d-2127-44e4-a892-8528813e83bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:47.587094 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c284f0d-2127-44e4-a892-8528813e83bf admin admin] PATH_INFO: `/v3/projects/76976ba899244faabedf1ddeff9e9f05/users/025eb41a391f41fd9a59d3b838b3b1b8/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:47.609603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c284f0d-2127-44e4-a892-8528813e83bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=76976ba899244faabedf1ddeff9e9f05, user_id=025eb41a391f41fd9a59d3b838b3b1b8, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:47.610650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-0c284f0d-2127-44e4-a892-8528813e83bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:48:47.610650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Sep 08 13:48:47.610650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: Sep 08 13:48:47.612560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c284f0d-2127-44e4-a892-8528813e83bf admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:47.636186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 754/1501] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:48:47 2022] PUT /identity/v3/projects/76976ba899244faabedf1ddeff9e9f05/users/025eb41a391f41fd9a59d3b838b3b1b8/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:48:47.649028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3963f-e163-45ee-b403-0cbf1387f9f7 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:47.649311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3963f-e163-45ee-b403-0cbf1387f9f7 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:47.649587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d3963f-e163-45ee-b403-0cbf1387f9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:47.707554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-59d3963f-e163-45ee-b403-0cbf1387f9f7 None None] MFA Rules not processed for user `025eb41a391f41fd9a59d3b838b3b1b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:48:47.708671 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-59d3963f-e163-45ee-b403-0cbf1387f9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:47.785205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 748/1502] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:48:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:48:47.801310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62eef5-3a0b-475c-9f85-97c91929f6fe None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:47.801599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62eef5-3a0b-475c-9f85-97c91929f6fe None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:47.801909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62eef5-3a0b-475c-9f85-97c91929f6fe None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:47.803338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 755/1503] 10.222.0.55 () {54 vars in 964 bytes} [Thu Sep 8 13:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:48:47.808253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38033b2-a688-4b70-99d7-26b910caa6ee None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:47.808537 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38033b2-a688-4b70-99d7-26b910caa6ee None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:47.808814 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38033b2-a688-4b70-99d7-26b910caa6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:47.842394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:47.887651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-c38033b2-a688-4b70-99d7-26b910caa6ee None None] MFA Rules not processed for user `ff93243f0c4c48d1924ea04a87a0e7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:48:47.888859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c38033b2-a688-4b70-99d7-26b910caa6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:47.966745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 749/1504] 10.222.0.55 () {58 vars in 1061 bytes} [Thu Sep 8 13:48:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:48:47.974646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62eef5-3a0b-475c-9f85-97c91929f6fe None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:47.976222 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2c62eef5-3a0b-475c-9f85-97c91929f6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:48.028826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:48.083397 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62eef5-3a0b-475c-9f85-97c91929f6fe None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:48.155862 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1dfc1-090a-4df7-a00d-910e6130926b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:48.156466 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1dfc1-090a-4df7-a00d-910e6130926b service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:48.156748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1dfc1-090a-4df7-a00d-910e6130926b service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:48.157023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a1dfc1-090a-4df7-a00d-910e6130926b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:48.158683 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-10a1dfc1-090a-4df7-a00d-910e6130926b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:48.256613 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1dfc1-090a-4df7-a00d-910e6130926b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:48.257984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a1dfc1-090a-4df7-a00d-910e6130926b service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:48.261347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:48.276681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 756/1505] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:48:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:48:48.322655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f63ff-3fe3-44d1-b967-9ae2b87170cf None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:48.323107 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f63ff-3fe3-44d1-b967-9ae2b87170cf None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:48.323214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f63ff-3fe3-44d1-b967-9ae2b87170cf None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:48.324525 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 750/1506] 10.222.0.55 () {52 vars in 830 bytes} [Thu Sep 8 13:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:48:48.328284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1dfc1-090a-4df7-a00d-910e6130926b service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:48.329495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-10a1dfc1-090a-4df7-a00d-910e6130926b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:48.427406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a1dfc1-090a-4df7-a00d-910e6130926b service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:48.522498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:48.523149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:48.523307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:48.523583 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] PATH_INFO: `/v3/users/025eb41a391f41fd9a59d3b838b3b1b8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:48.536440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=025eb41a391f41fd9a59d3b838b3b1b8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:48.537450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:48:48.537450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Sep 08 13:48:48.537450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: Sep 08 13:48:48.538698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:48.548191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:48.604369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '025eb41a391f41fd9a59d3b838b3b1b8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:48:48.614337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '025eb41a391f41fd9a59d3b838b3b1b8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:48:48.614919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '025eb41a391f41fd9a59d3b838b3b1b8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:48:48.615487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '025eb41a391f41fd9a59d3b838b3b1b8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:48:48.629800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '025eb41a391f41fd9a59d3b838b3b1b8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:48:48.659426 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 757/1507] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:48:48 2022] DELETE /identity/v3/users/025eb41a391f41fd9a59d3b838b3b1b8 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:48:48.663884 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6d2bc-9a4b-4956-a928-7a6530dcdc74 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:48.664170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6d2bc-9a4b-4956-a928-7a6530dcdc74 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:48.664443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6d2bc-9a4b-4956-a928-7a6530dcdc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:48.721884 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-21f6d2bc-9a4b-4956-a928-7a6530dcdc74 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:48:48.723068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-21f6d2bc-9a4b-4956-a928-7a6530dcdc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:48.798382 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:48.812409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 751/1508] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:48:48.822585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:48.823865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:48.926151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8567b76b-8561-49f4-8427-68217b63a3ed admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:48.983621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:49.002993 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a25a8a-ce83-447f-ac47-3becde4c1dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:49.003607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a25a8a-ce83-447f-ac47-3becde4c1dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:49.004030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a25a8a-ce83-447f-ac47-3becde4c1dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:49.004145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a25a8a-ce83-447f-ac47-3becde4c1dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:49.005754 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-68a25a8a-ce83-447f-ac47-3becde4c1dd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:49.114130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a25a8a-ce83-447f-ac47-3becde4c1dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:49.115586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a25a8a-ce83-447f-ac47-3becde4c1dd9 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:49.133238 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 758/1509] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:48:49.462426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6d2bc-9a4b-4956-a928-7a6530dcdc74 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:48:49.463669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-21f6d2bc-9a4b-4956-a928-7a6530dcdc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:48:49.479877 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6d2bc-9a4b-4956-a928-7a6530dcdc74 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:48:49.584320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28cf8f9-83df-402f-afc4-496b1e77472b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:48:49.584956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cf8f9-83df-402f-afc4-496b1e77472b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:48:49.585231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cf8f9-83df-402f-afc4-496b1e77472b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:48:49.585503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cf8f9-83df-402f-afc4-496b1e77472b admin admin] PATH_INFO: `/v3/projects/76976ba899244faabedf1ddeff9e9f05` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:48:49.595412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28cf8f9-83df-402f-afc4-496b1e77472b admin admin] RBAC: Authorizing `identity:delete_project(project_id=76976ba899244faabedf1ddeff9e9f05)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:48:49.596433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: WARNING py.warnings [None req-c28cf8f9-83df-402f-afc4-496b1e77472b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:48:49.596433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: warnings.warn(msg) Sep 08 13:48:49.596433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: Sep 08 13:48:49.597635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28cf8f9-83df-402f-afc4-496b1e77472b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:48:49.656219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c28cf8f9-83df-402f-afc4-496b1e77472b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76976ba899244faabedf1ddeff9e9f05', 'request_id': 'req-c28cf8f9-83df-402f-afc4-496b1e77472b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:48:49.669965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c28cf8f9-83df-402f-afc4-496b1e77472b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76976ba899244faabedf1ddeff9e9f05', 'request_id': 'req-c28cf8f9-83df-402f-afc4-496b1e77472b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:48:49.682821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c28cf8f9-83df-402f-afc4-496b1e77472b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76976ba899244faabedf1ddeff9e9f05', 'request_id': 'req-c28cf8f9-83df-402f-afc4-496b1e77472b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:48:49.706722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c28cf8f9-83df-402f-afc4-496b1e77472b admin admin] The token cache is being invalidate because project 76976ba899244faabedf1ddeff9e9f05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:48:49.706942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c28cf8f9-83df-402f-afc4-496b1e77472b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:48:49.707369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c28cf8f9-83df-402f-afc4-496b1e77472b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:48:49.708797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 752/1510] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:48:49 2022] DELETE /identity/v3/projects/76976ba899244faabedf1ddeff9e9f05 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:48:59.131030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=62 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:59.704105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:50:25.431320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0dc80-5df0-4c63-b7e9-114f69edd14d None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:25.431916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0dc80-5df0-4c63-b7e9-114f69edd14d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:25.431916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0dc80-5df0-4c63-b7e9-114f69edd14d None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:25.433279 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:50:25.438562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5c576-6f6e-4c69-bb89-aad28c49501c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:25.439029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5c576-6f6e-4c69-bb89-aad28c49501c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:25.439136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5c576-6f6e-4c69-bb89-aad28c49501c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:25.518578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-e8b5c576-6f6e-4c69-bb89-aad28c49501c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:50:25.519768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e8b5c576-6f6e-4c69-bb89-aad28c49501c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:25.620574 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:50:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:50:25.773797 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0dc80-5df0-4c63-b7e9-114f69edd14d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:50:25.775978 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e9f0dc80-5df0-4c63-b7e9-114f69edd14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:25.896982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0dc80-5df0-4c63-b7e9-114f69edd14d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:50:25.904754 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c8a1b2-10f9-4afd-b531-2683ca181709 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:50:25.905388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8a1b2-10f9-4afd-b531-2683ca181709 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:25.905666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8a1b2-10f9-4afd-b531-2683ca181709 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:25.905987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c8a1b2-10f9-4afd-b531-2683ca181709 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:25.916350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8a1b2-10f9-4afd-b531-2683ca181709 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:50:25.918092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c8a1b2-10f9-4afd-b531-2683ca181709 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:50:25.928367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 760/1513] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:50:27.033071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ce38d-156f-47f9-96a3-71fd0b62a4c4 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:27.033618 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ce38d-156f-47f9-96a3-71fd0b62a4c4 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:27.033618 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ce38d-156f-47f9-96a3-71fd0b62a4c4 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:27.034941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:50:27.041372 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfa3b8-569a-4f31-b8ad-822cb8205567 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:27.041810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfa3b8-569a-4f31-b8ad-822cb8205567 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:27.041977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfa3b8-569a-4f31-b8ad-822cb8205567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:27.061955 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7bcfa3b8-569a-4f31-b8ad-822cb8205567 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:50:27.063143 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7bcfa3b8-569a-4f31-b8ad-822cb8205567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:27.070890 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:50:27.226312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ce38d-156f-47f9-96a3-71fd0b62a4c4 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:50:27.240384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ce38d-156f-47f9-96a3-71fd0b62a4c4 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:50:27.245856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec119c-5eb4-4eae-b7bf-65d242654b22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:50:27.246453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec119c-5eb4-4eae-b7bf-65d242654b22 service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:27.246759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec119c-5eb4-4eae-b7bf-65d242654b22 service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:27.247035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec119c-5eb4-4eae-b7bf-65d242654b22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:27.255085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec119c-5eb4-4eae-b7bf-65d242654b22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:50:27.256671 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec119c-5eb4-4eae-b7bf-65d242654b22 service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:50:27.266469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 755/1516] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:50:28.627350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb26fd3-d7a3-4ede-be1d-b110f6fe6dad None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:28.627940 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb26fd3-d7a3-4ede-be1d-b110f6fe6dad None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:28.627940 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb26fd3-d7a3-4ede-be1d-b110f6fe6dad None None] PATH_INFO: `/v3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:28.629210 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 762/1517] 10.222.0.20 () {54 vars in 925 bytes} [Thu Sep 8 13:50:28 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:50:28.638194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d51b25-e28c-499a-a131-59381b929e97 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:28.638602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d51b25-e28c-499a-a131-59381b929e97 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:28.638790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d51b25-e28c-499a-a131-59381b929e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:28.698220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-21d51b25-e28c-499a-a131-59381b929e97 None None] MFA Rules not processed for user `e42fb05dbae64c758098c591d6ffa790`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:50:28.699928 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-21d51b25-e28c-499a-a131-59381b929e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:28.717184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:50:28.774345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 756/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Sep 8 13:50:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:50:37.057067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=56 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:50:38.292907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9715b4-dfa1-4e46-8716-a1456427dc8e None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:38.293468 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9715b4-dfa1-4e46-8716-a1456427dc8e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:38.293686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9715b4-dfa1-4e46-8716-a1456427dc8e None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:38.295124 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 763/1519] 10.222.0.55 () {54 vars in 961 bytes} [Thu Sep 8 13:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:50:38.300138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae1e45-f67f-4e5e-89f2-b52c88830fce None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:38.300587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae1e45-f67f-4e5e-89f2-b52c88830fce None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:38.300708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae1e45-f67f-4e5e-89f2-b52c88830fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:38.320031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-ceae1e45-f67f-4e5e-89f2-b52c88830fce None None] MFA Rules not processed for user `e42fb05dbae64c758098c591d6ffa790`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:50:38.321216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ceae1e45-f67f-4e5e-89f2-b52c88830fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:38.330803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 757/1520] 10.222.0.55 () {58 vars in 1058 bytes} [Thu Sep 8 13:50:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:50:38.338895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9715b4-dfa1-4e46-8716-a1456427dc8e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:50:38.353028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9715b4-dfa1-4e46-8716-a1456427dc8e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:50:38.357964 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995024b9-af10-49d8-8c8e-19928a7b85ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:50:38.358564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995024b9-af10-49d8-8c8e-19928a7b85ce service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:38.358839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995024b9-af10-49d8-8c8e-19928a7b85ce service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:38.359114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995024b9-af10-49d8-8c8e-19928a7b85ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:38.367625 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995024b9-af10-49d8-8c8e-19928a7b85ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:50:38.369236 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995024b9-af10-49d8-8c8e-19928a7b85ce service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:50:38.378876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 764/1521] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:50:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:50:38.739609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7f0df-ef73-4806-bfa5-f652ecfb6ab3 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:38.739854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7f0df-ef73-4806-bfa5-f652ecfb6ab3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:38.740138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7f0df-ef73-4806-bfa5-f652ecfb6ab3 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:38.741505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:50:38.748265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc5c0a-296c-430a-96f4-3e0d6e406e71 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:38.748559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc5c0a-296c-430a-96f4-3e0d6e406e71 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:38.748836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc5c0a-296c-430a-96f4-3e0d6e406e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:38.769671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-83dc5c0a-296c-430a-96f4-3e0d6e406e71 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:50:38.770898 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-83dc5c0a-296c-430a-96f4-3e0d6e406e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:38.778804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:50:38.938915 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7f0df-ef73-4806-bfa5-f652ecfb6ab3 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:50:38.953567 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7f0df-ef73-4806-bfa5-f652ecfb6ab3 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:50:38.960627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93ec74-e1af-430f-9659-0365734107bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:50:38.961304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93ec74-e1af-430f-9659-0365734107bb service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:38.961604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93ec74-e1af-430f-9659-0365734107bb service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:38.961996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93ec74-e1af-430f-9659-0365734107bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:38.970678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93ec74-e1af-430f-9659-0365734107bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:50:38.972312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93ec74-e1af-430f-9659-0365734107bb service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:50:38.982805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 759/1524] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:50:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:50:40.048577 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5e0a2-5bee-433f-a05b-95a8c92f79ac None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:40.049137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5e0a2-5bee-433f-a05b-95a8c92f79ac None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:40.049137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5e0a2-5bee-433f-a05b-95a8c92f79ac None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:40.050514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:50:40.056486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee24c6e-3876-4cbb-9830-be377cb5f81f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:40.057020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee24c6e-3876-4cbb-9830-be377cb5f81f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:40.057020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee24c6e-3876-4cbb-9830-be377cb5f81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:40.076374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-cee24c6e-3876-4cbb-9830-be377cb5f81f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:50:40.077560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-cee24c6e-3876-4cbb-9830-be377cb5f81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:40.085608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:50:40.235839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5e0a2-5bee-433f-a05b-95a8c92f79ac None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:50:40.250495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5e0a2-5bee-433f-a05b-95a8c92f79ac None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:50:40.255829 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b763ef96-18b0-4b32-8873-3a5a377b2d28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:50:40.256420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763ef96-18b0-4b32-8873-3a5a377b2d28 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:40.256696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763ef96-18b0-4b32-8873-3a5a377b2d28 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:40.256968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763ef96-18b0-4b32-8873-3a5a377b2d28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:40.265424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763ef96-18b0-4b32-8873-3a5a377b2d28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:50:40.267030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b763ef96-18b0-4b32-8873-3a5a377b2d28 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:50:40.276323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 767/1527] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:50:44.566830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64084f4b-fd85-41cb-b881-64fbeb271bb0 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:44.567425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64084f4b-fd85-41cb-b881-64fbeb271bb0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:44.567583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64084f4b-fd85-41cb-b881-64fbeb271bb0 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:44.569191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:50:44.574362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200605a5-1e11-46a0-8b98-7c77145dfd79 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:44.574785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200605a5-1e11-46a0-8b98-7c77145dfd79 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:44.574897 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200605a5-1e11-46a0-8b98-7c77145dfd79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:44.594347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-200605a5-1e11-46a0-8b98-7c77145dfd79 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:50:44.595535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-200605a5-1e11-46a0-8b98-7c77145dfd79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:44.603379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:50:44.891423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64084f4b-fd85-41cb-b881-64fbeb271bb0 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:50:44.893230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-64084f4b-fd85-41cb-b881-64fbeb271bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:44.974917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64084f4b-fd85-41cb-b881-64fbeb271bb0 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:50:44.978509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9995654e-dc76-4b55-950a-25cd60d36870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:50:44.979111 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995654e-dc76-4b55-950a-25cd60d36870 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:44.979388 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995654e-dc76-4b55-950a-25cd60d36870 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:44.979660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9995654e-dc76-4b55-950a-25cd60d36870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:44.988924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995654e-dc76-4b55-950a-25cd60d36870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:50:44.990382 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9995654e-dc76-4b55-950a-25cd60d36870 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:50:44.999687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 762/1530] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:50:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:50:47.144805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1ae1d-85d3-481a-b513-38d57387d128 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:47.145325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1ae1d-85d3-481a-b513-38d57387d128 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:47.145325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1ae1d-85d3-481a-b513-38d57387d128 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:47.146763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:50:47.151938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe011d-ae4b-43e9-8c2c-b6c6fde6e987 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:47.152876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe011d-ae4b-43e9-8c2c-b6c6fde6e987 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:47.152982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe011d-ae4b-43e9-8c2c-b6c6fde6e987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:47.173752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-0abe011d-ae4b-43e9-8c2c-b6c6fde6e987 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:50:47.174946 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0abe011d-ae4b-43e9-8c2c-b6c6fde6e987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:47.182676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:50:47.476048 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1ae1d-85d3-481a-b513-38d57387d128 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:50:47.487010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1ae1d-85d3-481a-b513-38d57387d128 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:50:47.490466 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2bc78-ef43-4783-b867-2d9c5841a01b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:50:47.491056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2bc78-ef43-4783-b867-2d9c5841a01b service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:47.491332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2bc78-ef43-4783-b867-2d9c5841a01b service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:47.491602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2bc78-ef43-4783-b867-2d9c5841a01b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:47.500721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2bc78-ef43-4783-b867-2d9c5841a01b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:50:47.502167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2bc78-ef43-4783-b867-2d9c5841a01b service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:50:47.511276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 770/1533] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:50:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:50:51.356267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab88d5-4802-446c-85aa-f405b69c13a0 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:51.356836 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab88d5-4802-446c-85aa-f405b69c13a0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:51.356836 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab88d5-4802-446c-85aa-f405b69c13a0 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:51.358232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:50:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:50:51.363297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5f7b6-f68e-4662-aead-be25199abf7f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:51.363695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5f7b6-f68e-4662-aead-be25199abf7f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:51.363836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5f7b6-f68e-4662-aead-be25199abf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:51.383110 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-3bb5f7b6-f68e-4662-aead-be25199abf7f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:50:51.384294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3bb5f7b6-f68e-4662-aead-be25199abf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:51.392603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:50:51.705511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab88d5-4802-446c-85aa-f405b69c13a0 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:50:51.707494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-64ab88d5-4802-446c-85aa-f405b69c13a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:51.771057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:50:51.786291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ab88d5-4802-446c-85aa-f405b69c13a0 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:50:51.789827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698a674-e17c-4d5b-a264-1c4d8a391285 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:50:51.790416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698a674-e17c-4d5b-a264-1c4d8a391285 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:51.790689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698a674-e17c-4d5b-a264-1c4d8a391285 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:51.790958 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698a674-e17c-4d5b-a264-1c4d8a391285 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:51.800191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698a674-e17c-4d5b-a264-1c4d8a391285 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:50:51.801593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698a674-e17c-4d5b-a264-1c4d8a391285 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:50:51.811419 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 765/1536] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:50:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:50:53.961422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54ca9c-5560-444c-b721-11125fb59af6 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:53.962011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54ca9c-5560-444c-b721-11125fb59af6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:53.962011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54ca9c-5560-444c-b721-11125fb59af6 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:53.963335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:50:53.968518 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535b6d0-5dd2-4783-80ff-d5980792e601 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:53.968922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535b6d0-5dd2-4783-80ff-d5980792e601 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:53.969070 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535b6d0-5dd2-4783-80ff-d5980792e601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:53.988444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b535b6d0-5dd2-4783-80ff-d5980792e601 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:50:53.989635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b535b6d0-5dd2-4783-80ff-d5980792e601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:53.997922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:50:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:50:54.301541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54ca9c-5560-444c-b721-11125fb59af6 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:50:54.303223 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6e54ca9c-5560-444c-b721-11125fb59af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:50:54.316846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54ca9c-5560-444c-b721-11125fb59af6 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:50:54.320346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9d436-0864-4973-b4b3-ca502b685ba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:50:54.320935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9d436-0864-4973-b4b3-ca502b685ba0 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:50:54.321213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9d436-0864-4973-b4b3-ca502b685ba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:50:54.321483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9d436-0864-4973-b4b3-ca502b685ba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:50:54.330192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9d436-0864-4973-b4b3-ca502b685ba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:50:54.331607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9d436-0864-4973-b4b3-ca502b685ba0 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:50:54.341072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 773/1539] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:51:03.983544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=11 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:04.337065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=33 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:20.491961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588c6f4-8e39-401e-a528-b89208677c73 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.492622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588c6f4-8e39-401e-a528-b89208677c73 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.492622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588c6f4-8e39-401e-a528-b89208677c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.518647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4588c6f4-8e39-401e-a528-b89208677c73 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:20.520314 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4588c6f4-8e39-401e-a528-b89208677c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:20.534883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 767/1540] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:20.544780 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b0923-5107-41bf-b592-1a0363c66c04 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.545877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b0923-5107-41bf-b592-1a0363c66c04 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.546920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b0923-5107-41bf-b592-1a0363c66c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.573262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4a4b0923-5107-41bf-b592-1a0363c66c04 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:20.574979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4a4b0923-5107-41bf-b592-1a0363c66c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:20.588838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 774/1541] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:20.594156 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588c6f4-8e39-401e-a528-b89208677c73 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:20.611947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588c6f4-8e39-401e-a528-b89208677c73 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:20.620700 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85ec80-8646-4a85-b691-a2e5e33ca02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:20.621776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85ec80-8646-4a85-b691-a2e5e33ca02b admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.622490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85ec80-8646-4a85-b691-a2e5e33ca02b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.622893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85ec80-8646-4a85-b691-a2e5e33ca02b admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.623817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85ec80-8646-4a85-b691-a2e5e33ca02b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:20.624410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85ec80-8646-4a85-b691-a2e5e33ca02b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:20.626595 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85ec80-8646-4a85-b691-a2e5e33ca02b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:20.645098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 768/1542] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:20.653355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b0923-5107-41bf-b592-1a0363c66c04 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:20.668328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b0923-5107-41bf-b592-1a0363c66c04 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:20.679106 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a10e8-b279-4236-8355-bc58da04c4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:20.680076 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a10e8-b279-4236-8355-bc58da04c4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.680580 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a10e8-b279-4236-8355-bc58da04c4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.681093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a10e8-b279-4236-8355-bc58da04c4b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.681992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a10e8-b279-4236-8355-bc58da04c4b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:20.684435 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a10e8-b279-4236-8355-bc58da04c4b5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:20.728870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 775/1543] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:20 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:20.733067 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85ec80-8646-4a85-b691-a2e5e33ca02b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:20.747283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85ec80-8646-4a85-b691-a2e5e33ca02b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:20.828698 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b322f-b812-486f-9ce2-cce5d7facee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:20.829325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b322f-b812-486f-9ce2-cce5d7facee7 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.829774 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b322f-b812-486f-9ce2-cce5d7facee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.830192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b322f-b812-486f-9ce2-cce5d7facee7 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.831099 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b322f-b812-486f-9ce2-cce5d7facee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:20.833188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b322f-b812-486f-9ce2-cce5d7facee7 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:20.845868 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa9701-e55a-4ebd-b948-36c027d045a8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.846222 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa9701-e55a-4ebd-b948-36c027d045a8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.846482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa9701-e55a-4ebd-b948-36c027d045a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.875464 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 769/1544] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:20 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:20.879832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b322f-b812-486f-9ce2-cce5d7facee7 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:20.884087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-82fa9701-e55a-4ebd-b948-36c027d045a8 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:20.885357 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-82fa9701-e55a-4ebd-b948-36c027d045a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:20.892928 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b322f-b812-486f-9ce2-cce5d7facee7 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:20.907085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 776/1545] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:52:20.907472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c3294-1c4d-4b17-a24b-dd01988f3d5a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.907684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c3294-1c4d-4b17-a24b-dd01988f3d5a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.907966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c3294-1c4d-4b17-a24b-dd01988f3d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.911645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edd647-c648-4843-ad2b-3eb10d19de6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:20.911645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edd647-c648-4843-ad2b-3eb10d19de6d admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.911645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edd647-c648-4843-ad2b-3eb10d19de6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.911645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edd647-c648-4843-ad2b-3eb10d19de6d admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.912687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edd647-c648-4843-ad2b-3eb10d19de6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:20.913936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edd647-c648-4843-ad2b-3eb10d19de6d admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:20.939947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-560c3294-1c4d-4b17-a24b-dd01988f3d5a None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:20.941273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-560c3294-1c4d-4b17-a24b-dd01988f3d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:20.942540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 770/1546] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:20.944380 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc1ce2-95d3-4903-991e-4927c1c6bee6 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.944666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc1ce2-95d3-4903-991e-4927c1c6bee6 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.945032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc1ce2-95d3-4903-991e-4927c1c6bee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.952112 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 777/1547] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:20.953591 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6be48-363f-4dcf-b89f-c848641eb668 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.954105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6be48-363f-4dcf-b89f-c848641eb668 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.954390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6be48-363f-4dcf-b89f-c848641eb668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.968699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-c0dc1ce2-95d3-4903-991e-4927c1c6bee6 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:20.970901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c0dc1ce2-95d3-4903-991e-4927c1c6bee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:20.983213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 771/1548] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:20.984550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415eef7-66d9-40a1-befd-db95f71bafcb None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.984837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415eef7-66d9-40a1-befd-db95f71bafcb None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.985112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415eef7-66d9-40a1-befd-db95f71bafcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:20.985356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-33a6be48-363f-4dcf-b89f-c848641eb668 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:20.987653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-33a6be48-363f-4dcf-b89f-c848641eb668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:20.996144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 778/1549] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:20.997462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22163037-79f6-4d54-b96f-700560e3c8db None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:20.997685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22163037-79f6-4d54-b96f-700560e3c8db None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:20.997980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22163037-79f6-4d54-b96f-700560e3c8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.007681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-5415eef7-66d9-40a1-befd-db95f71bafcb None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:21.008878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5415eef7-66d9-40a1-befd-db95f71bafcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:21.017586 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 772/1550] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:21.018431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415eef7-66d9-40a1-befd-db95f71bafcb None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.018900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-22163037-79f6-4d54-b96f-700560e3c8db None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:21.020082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-22163037-79f6-4d54-b96f-700560e3c8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:21.027864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415eef7-66d9-40a1-befd-db95f71bafcb None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.028289 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 779/1551] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:21.029475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44edc5c-6d3d-4303-b461-bf63a079ac0f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.029789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44edc5c-6d3d-4303-b461-bf63a079ac0f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.030053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44edc5c-6d3d-4303-b461-bf63a079ac0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.032870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af7ebe-7213-4a9c-b216-82086bace096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.033467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af7ebe-7213-4a9c-b216-82086bace096 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.033770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af7ebe-7213-4a9c-b216-82086bace096 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.034040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af7ebe-7213-4a9c-b216-82086bace096 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.034687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af7ebe-7213-4a9c-b216-82086bace096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.036278 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af7ebe-7213-4a9c-b216-82086bace096 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.049719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 773/1552] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:21.049719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1a487-895c-4f07-9500-9c59359ac993 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.049719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1a487-895c-4f07-9500-9c59359ac993 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.049719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1a487-895c-4f07-9500-9c59359ac993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.050671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d44edc5c-6d3d-4303-b461-bf63a079ac0f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:21.051913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d44edc5c-6d3d-4303-b461-bf63a079ac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:21.060111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 780/1553] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:21.061136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceac5fa-590b-4a00-a398-d99df6eb617c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.061424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceac5fa-590b-4a00-a398-d99df6eb617c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.061702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceac5fa-590b-4a00-a398-d99df6eb617c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.070452 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-5df1a487-895c-4f07-9500-9c59359ac993 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:21.071654 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5df1a487-895c-4f07-9500-9c59359ac993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:21.079587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 774/1554] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:21.080755 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7582ecc-8ca9-4ce8-a21b-98b01aacf658 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.081046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7582ecc-8ca9-4ce8-a21b-98b01aacf658 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.081318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7582ecc-8ca9-4ce8-a21b-98b01aacf658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.081444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-cceac5fa-590b-4a00-a398-d99df6eb617c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:21.082573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cceac5fa-590b-4a00-a398-d99df6eb617c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:21.091902 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 781/1555] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:21.092963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a905e0f-7be7-468a-b1a5-405d7c3ad113 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.093815 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a905e0f-7be7-468a-b1a5-405d7c3ad113 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.094111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a905e0f-7be7-468a-b1a5-405d7c3ad113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.101437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-f7582ecc-8ca9-4ce8-a21b-98b01aacf658 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:21.102672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f7582ecc-8ca9-4ce8-a21b-98b01aacf658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:21.110649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 775/1556] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:21.111423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7582ecc-8ca9-4ce8-a21b-98b01aacf658 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.114150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8a905e0f-7be7-468a-b1a5-405d7c3ad113 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:21.115320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8a905e0f-7be7-468a-b1a5-405d7c3ad113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:21.120549 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7582ecc-8ca9-4ce8-a21b-98b01aacf658 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.123158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 782/1557] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:21.124188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d522-4005-4cd9-b948-ec438b84775d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.125005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d522-4005-4cd9-b948-ec438b84775d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.125283 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68d522-4005-4cd9-b948-ec438b84775d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.126285 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eeb622-8ba0-4e83-b7d2-a8704da16510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.126874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eeb622-8ba0-4e83-b7d2-a8704da16510 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.127153 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eeb622-8ba0-4e83-b7d2-a8704da16510 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.127420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eeb622-8ba0-4e83-b7d2-a8704da16510 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.128059 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eeb622-8ba0-4e83-b7d2-a8704da16510 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:21.128359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eeb622-8ba0-4e83-b7d2-a8704da16510 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.130006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eeb622-8ba0-4e83-b7d2-a8704da16510 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.141724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 776/1558] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:21.142593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eeb622-8ba0-4e83-b7d2-a8704da16510 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.145396 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5f68d522-4005-4cd9-b948-ec438b84775d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:21.146693 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5f68d522-4005-4cd9-b948-ec438b84775d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:21.152373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eeb622-8ba0-4e83-b7d2-a8704da16510 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.154625 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 783/1559] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:21.155416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68d522-4005-4cd9-b948-ec438b84775d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.157509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30d0fb-8ff2-465e-8cc3-03164b9b1646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.158176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30d0fb-8ff2-465e-8cc3-03164b9b1646 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.158456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30d0fb-8ff2-465e-8cc3-03164b9b1646 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.158849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30d0fb-8ff2-465e-8cc3-03164b9b1646 admin admin] PATH_INFO: `/v3/projects/8a4eb5c1f5b44cfab4a3869cc0a06851/users/cbea503b4cca4fb886179ff3e0564301/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.165124 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68d522-4005-4cd9-b948-ec438b84775d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.169981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcc816-7f48-482b-91d6-3ba2c4f14304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.170693 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcc816-7f48-482b-91d6-3ba2c4f14304 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.170996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcc816-7f48-482b-91d6-3ba2c4f14304 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.171334 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcc816-7f48-482b-91d6-3ba2c4f14304 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.171982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befcc816-7f48-482b-91d6-3ba2c4f14304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:21.172288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befcc816-7f48-482b-91d6-3ba2c4f14304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.173305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-befcc816-7f48-482b-91d6-3ba2c4f14304 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:52:21.173305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Sep 08 13:52:21.173305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: Sep 08 13:52:21.173582 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30d0fb-8ff2-465e-8cc3-03164b9b1646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a4eb5c1f5b44cfab4a3869cc0a06851, user_id=cbea503b4cca4fb886179ff3e0564301, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.174308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befcc816-7f48-482b-91d6-3ba2c4f14304 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.174422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: WARNING py.warnings [None req-dc30d0fb-8ff2-465e-8cc3-03164b9b1646 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:52:21.174422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: warnings.warn(msg) Sep 08 13:52:21.174422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: Sep 08 13:52:21.176350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30d0fb-8ff2-465e-8cc3-03164b9b1646 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.186481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 784/1560] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:21.188401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7059fc-2244-477b-a729-43552b8883d3 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.188865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7059fc-2244-477b-a729-43552b8883d3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.189138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7059fc-2244-477b-a729-43552b8883d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.192872 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 777/1561] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:21 2022] PUT /identity/v3/projects/8a4eb5c1f5b44cfab4a3869cc0a06851/users/cbea503b4cca4fb886179ff3e0564301/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:21.195044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde97604-0585-4a62-b165-f8a8245c41d4 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.195333 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde97604-0585-4a62-b165-f8a8245c41d4 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.195609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde97604-0585-4a62-b165-f8a8245c41d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.210358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-dc7059fc-2244-477b-a729-43552b8883d3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:21.211975 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dc7059fc-2244-477b-a729-43552b8883d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:21.216777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-bde97604-0585-4a62-b165-f8a8245c41d4 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:21.217974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-bde97604-0585-4a62-b165-f8a8245c41d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:21.260255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:21.275382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 785/1562] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:21.275382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7059fc-2244-477b-a729-43552b8883d3 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.281694 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7059fc-2244-477b-a729-43552b8883d3 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.284897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 778/1563] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:21.285558 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde97604-0585-4a62-b165-f8a8245c41d4 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.287400 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10e344-9580-4056-b932-2e9fc5586cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.288297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10e344-9580-4056-b932-2e9fc5586cdc admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.288578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10e344-9580-4056-b932-2e9fc5586cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.288849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10e344-9580-4056-b932-2e9fc5586cdc admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.289501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10e344-9580-4056-b932-2e9fc5586cdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:21.289844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10e344-9580-4056-b932-2e9fc5586cdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.291603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10e344-9580-4056-b932-2e9fc5586cdc admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.294819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde97604-0585-4a62-b165-f8a8245c41d4 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.299933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8283740-f08f-4577-9e2a-95c1441cf6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.300521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8283740-f08f-4577-9e2a-95c1441cf6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.300801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8283740-f08f-4577-9e2a-95c1441cf6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.301072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8283740-f08f-4577-9e2a-95c1441cf6e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.301783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8283740-f08f-4577-9e2a-95c1441cf6e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:21.302087 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8283740-f08f-4577-9e2a-95c1441cf6e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.303651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8283740-f08f-4577-9e2a-95c1441cf6e7 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.311681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 786/1564] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:21.311681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10e344-9580-4056-b932-2e9fc5586cdc admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.315659 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 779/1565] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:21.316330 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8283740-f08f-4577-9e2a-95c1441cf6e7 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.316572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10e344-9580-4056-b932-2e9fc5586cdc admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.326229 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05a244-cef7-42c5-9e54-f6085c964c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.326520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8283740-f08f-4577-9e2a-95c1441cf6e7 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.326838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05a244-cef7-42c5-9e54-f6085c964c69 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.327152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05a244-cef7-42c5-9e54-f6085c964c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.327965 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05a244-cef7-42c5-9e54-f6085c964c69 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.328600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05a244-cef7-42c5-9e54-f6085c964c69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.330575 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05a244-cef7-42c5-9e54-f6085c964c69 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.331769 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b780f-02fd-4e7d-9625-dc4a80d49105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.332347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b780f-02fd-4e7d-9625-dc4a80d49105 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.332628 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b780f-02fd-4e7d-9625-dc4a80d49105 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.332902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4b780f-02fd-4e7d-9625-dc4a80d49105 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.333535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b780f-02fd-4e7d-9625-dc4a80d49105 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:21.333865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b780f-02fd-4e7d-9625-dc4a80d49105 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.335680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4b780f-02fd-4e7d-9625-dc4a80d49105 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.348013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 780/1566] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:21.348830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b780f-02fd-4e7d-9625-dc4a80d49105 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.360291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 787/1567] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/projects => generated 405 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:21.363426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4b780f-02fd-4e7d-9625-dc4a80d49105 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.363585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81b02a-8718-4264-81b9-e5d44fe74aa3 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.363585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81b02a-8718-4264-81b9-e5d44fe74aa3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.363585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb81b02a-8718-4264-81b9-e5d44fe74aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.388618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-cb81b02a-8718-4264-81b9-e5d44fe74aa3 None None] MFA Rules not processed for user `cbea503b4cca4fb886179ff3e0564301`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:21.389849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-cb81b02a-8718-4264-81b9-e5d44fe74aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:21.399388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:21.418628 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5f975-0109-4fd3-b1fb-329bb0020229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.419247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5f975-0109-4fd3-b1fb-329bb0020229 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.419526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5f975-0109-4fd3-b1fb-329bb0020229 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.419797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5f975-0109-4fd3-b1fb-329bb0020229 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.420431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5f975-0109-4fd3-b1fb-329bb0020229 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.422023 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: WARNING py.warnings [None req-dec5f975-0109-4fd3-b1fb-329bb0020229 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:52:21.422023 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: warnings.warn(msg) Sep 08 13:52:21.422023 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: Sep 08 13:52:21.423252 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5f975-0109-4fd3-b1fb-329bb0020229 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.449972 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 781/1568] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:21.450803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5f975-0109-4fd3-b1fb-329bb0020229 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.458934 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 788/1569] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:21.464436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81b02a-8718-4264-81b9-e5d44fe74aa3 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.476732 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb81b02a-8718-4264-81b9-e5d44fe74aa3 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.523136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5f975-0109-4fd3-b1fb-329bb0020229 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.527911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ae515-382d-4b21-888d-28996a00d72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.528524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ae515-382d-4b21-888d-28996a00d72d admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.528803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ae515-382d-4b21-888d-28996a00d72d admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.529075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ae515-382d-4b21-888d-28996a00d72d admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.529722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ae515-382d-4b21-888d-28996a00d72d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:21.530051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ae515-382d-4b21-888d-28996a00d72d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.531703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ae515-382d-4b21-888d-28996a00d72d admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.543091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 782/1570] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:21.543882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ae515-382d-4b21-888d-28996a00d72d admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.545362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc7ed3-8a55-4fc7-892a-7e656f6de456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.546819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc7ed3-8a55-4fc7-892a-7e656f6de456 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.547103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc7ed3-8a55-4fc7-892a-7e656f6de456 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.547377 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc7ed3-8a55-4fc7-892a-7e656f6de456 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.548053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc7ed3-8a55-4fc7-892a-7e656f6de456 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:21.548470 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc7ed3-8a55-4fc7-892a-7e656f6de456 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.550878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc7ed3-8a55-4fc7-892a-7e656f6de456 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.553694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ae515-382d-4b21-888d-28996a00d72d admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.558324 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b23bba-6fd0-4084-b78a-1d21e6498c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.558907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b23bba-6fd0-4084-b78a-1d21e6498c1e admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.559186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b23bba-6fd0-4084-b78a-1d21e6498c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.559460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b23bba-6fd0-4084-b78a-1d21e6498c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.560076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b23bba-6fd0-4084-b78a-1d21e6498c1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.562033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b23bba-6fd0-4084-b78a-1d21e6498c1e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.562781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 789/1571] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:21.563561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc7ed3-8a55-4fc7-892a-7e656f6de456 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.575018 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc7ed3-8a55-4fc7-892a-7e656f6de456 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.580055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9c9ba-fb9a-4739-98db-d378b7e00244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.580624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9c9ba-fb9a-4739-98db-d378b7e00244 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.580967 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9c9ba-fb9a-4739-98db-d378b7e00244 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.581257 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9c9ba-fb9a-4739-98db-d378b7e00244 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.581933 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9c9ba-fb9a-4739-98db-d378b7e00244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.583981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9c9ba-fb9a-4739-98db-d378b7e00244 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.589455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 783/1572] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:21.590295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b23bba-6fd0-4084-b78a-1d21e6498c1e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.609763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:21.618169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 790/1573] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:21.618169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9c9ba-fb9a-4739-98db-d378b7e00244 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.657541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b23bba-6fd0-4084-b78a-1d21e6498c1e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.678057 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9c9ba-fb9a-4739-98db-d378b7e00244 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.682733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aab239-a63a-4556-9782-2c3de955bdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.683336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab239-a63a-4556-9782-2c3de955bdf8 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.683614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab239-a63a-4556-9782-2c3de955bdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.683887 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aab239-a63a-4556-9782-2c3de955bdf8 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.684516 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aab239-a63a-4556-9782-2c3de955bdf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.685527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-37aab239-a63a-4556-9782-2c3de955bdf8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:52:21.685527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Sep 08 13:52:21.685527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: Sep 08 13:52:21.687339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aab239-a63a-4556-9782-2c3de955bdf8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.709541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 791/1574] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:21.711106 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aab239-a63a-4556-9782-2c3de955bdf8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.714193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966b262-36e7-4720-a7c7-01004bfb023c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.714805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966b262-36e7-4720-a7c7-01004bfb023c admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.715083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966b262-36e7-4720-a7c7-01004bfb023c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.715356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966b262-36e7-4720-a7c7-01004bfb023c admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.715983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966b262-36e7-4720-a7c7-01004bfb023c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.717900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966b262-36e7-4720-a7c7-01004bfb023c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.721223 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aab239-a63a-4556-9782-2c3de955bdf8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.725863 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33e351-65eb-4957-ad25-3e292931859a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.726456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33e351-65eb-4957-ad25-3e292931859a admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.726739 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33e351-65eb-4957-ad25-3e292931859a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.727013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33e351-65eb-4957-ad25-3e292931859a admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.727639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33e351-65eb-4957-ad25-3e292931859a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.729529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33e351-65eb-4957-ad25-3e292931859a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.740936 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:21.745801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 784/1575] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/projects => generated 371 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:21.747039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff9c35-5fba-4a83-bfd6-f3ef5e3906d2 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.747329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff9c35-5fba-4a83-bfd6-f3ef5e3906d2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.747607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff9c35-5fba-4a83-bfd6-f3ef5e3906d2 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.749233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 785/1576] 10.222.0.55 () {54 vars in 962 bytes} [Thu Sep 8 13:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:21.750421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff9c35-5fba-4a83-bfd6-f3ef5e3906d2 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.756185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 792/1577] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:21.757196 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33e351-65eb-4957-ad25-3e292931859a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.767190 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33e351-65eb-4957-ad25-3e292931859a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.814576 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff9c35-5fba-4a83-bfd6-f3ef5e3906d2 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.819061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0222eb-844e-41fc-9d8c-c5538ee9e1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.819660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0222eb-844e-41fc-9d8c-c5538ee9e1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.819939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0222eb-844e-41fc-9d8c-c5538ee9e1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.820217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0222eb-844e-41fc-9d8c-c5538ee9e1b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.820847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0222eb-844e-41fc-9d8c-c5538ee9e1b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.821642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5da4b-8d79-4e90-bc7d-58ebc027ec9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.822278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5da4b-8d79-4e90-bc7d-58ebc027ec9b admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.822568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5da4b-8d79-4e90-bc7d-58ebc027ec9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.822789 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0222eb-844e-41fc-9d8c-c5538ee9e1b0 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.822904 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5da4b-8d79-4e90-bc7d-58ebc027ec9b admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.823438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5da4b-8d79-4e90-bc7d-58ebc027ec9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.825437 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5da4b-8d79-4e90-bc7d-58ebc027ec9b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.853236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 786/1578] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:21.857081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 793/1579] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:21.857235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0222eb-844e-41fc-9d8c-c5538ee9e1b0 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.857330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5da4b-8d79-4e90-bc7d-58ebc027ec9b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.892158 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:21.918611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5da4b-8d79-4e90-bc7d-58ebc027ec9b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.923422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0222eb-844e-41fc-9d8c-c5538ee9e1b0 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.924250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3273f5-87e6-4e3c-8d4e-7ba03d5e924d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.924875 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3273f5-87e6-4e3c-8d4e-7ba03d5e924d admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.925175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3273f5-87e6-4e3c-8d4e-7ba03d5e924d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.925510 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3273f5-87e6-4e3c-8d4e-7ba03d5e924d admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.926173 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3273f5-87e6-4e3c-8d4e-7ba03d5e924d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.928111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3273f5-87e6-4e3c-8d4e-7ba03d5e924d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.928237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c56f5-7f4a-43a6-9e69-05c2af5dbf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.928760 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c56f5-7f4a-43a6-9e69-05c2af5dbf20 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.929038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c56f5-7f4a-43a6-9e69-05c2af5dbf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.929311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c56f5-7f4a-43a6-9e69-05c2af5dbf20 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.930000 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c56f5-7f4a-43a6-9e69-05c2af5dbf20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.931917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c56f5-7f4a-43a6-9e69-05c2af5dbf20 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.950736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 794/1580] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:21.951593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3273f5-87e6-4e3c-8d4e-7ba03d5e924d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.955998 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 787/1581] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:21.956800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c56f5-7f4a-43a6-9e69-05c2af5dbf20 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:21.964211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3273f5-87e6-4e3c-8d4e-7ba03d5e924d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.966742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c56f5-7f4a-43a6-9e69-05c2af5dbf20 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:21.968888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4cbc95-2d11-4448-b7e2-6c1d90c8e87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.969500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cbc95-2d11-4448-b7e2-6c1d90c8e87a admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.969812 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cbc95-2d11-4448-b7e2-6c1d90c8e87a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.970088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cbc95-2d11-4448-b7e2-6c1d90c8e87a admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.970726 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cbc95-2d11-4448-b7e2-6c1d90c8e87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.972121 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b1302-5445-4ba8-938f-92f3404e3b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:21.972316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4cbc95-2d11-4448-b7e2-6c1d90c8e87a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.972712 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b1302-5445-4ba8-938f-92f3404e3b17 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.972995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b1302-5445-4ba8-938f-92f3404e3b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.973270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b1302-5445-4ba8-938f-92f3404e3b17 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:21.973932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b1302-5445-4ba8-938f-92f3404e3b17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:21.975849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b1302-5445-4ba8-938f-92f3404e3b17 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:21.982513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 795/1582] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:21.984189 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbaa55-4779-4b5c-8244-fd6ec3b0ee5f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:21.984475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbaa55-4779-4b5c-8244-fd6ec3b0ee5f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:21.984861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbaa55-4779-4b5c-8244-fd6ec3b0ee5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.000435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 788/1583] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/users => generated 286 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:22.001339 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b1302-5445-4ba8-938f-92f3404e3b17 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.010865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b1302-5445-4ba8-938f-92f3404e3b17 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.016075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a21fe7-bb27-41e6-9919-b0b078f77e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.016701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a21fe7-bb27-41e6-9919-b0b078f77e4a admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.016933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a21fe7-bb27-41e6-9919-b0b078f77e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.017208 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a21fe7-bb27-41e6-9919-b0b078f77e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.017887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a21fe7-bb27-41e6-9919-b0b078f77e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.019494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a21fe7-bb27-41e6-9919-b0b078f77e4a admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.027911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4fbbaa55-4779-4b5c-8244-fd6ec3b0ee5f None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:22.029224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4fbbaa55-4779-4b5c-8244-fd6ec3b0ee5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:22.030336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 789/1584] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:22.031190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a21fe7-bb27-41e6-9919-b0b078f77e4a admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.041290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a21fe7-bb27-41e6-9919-b0b078f77e4a admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.045896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc22f8-561a-4e45-ae8b-983a1cf249fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.048514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc22f8-561a-4e45-ae8b-983a1cf249fa admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.048789 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc22f8-561a-4e45-ae8b-983a1cf249fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.050052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc22f8-561a-4e45-ae8b-983a1cf249fa admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.052181 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc22f8-561a-4e45-ae8b-983a1cf249fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.055240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc22f8-561a-4e45-ae8b-983a1cf249fa admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.079949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:22.084109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 790/1585] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:22 2022] POST /identity/v3/users => generated 289 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:22.085513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc22f8-561a-4e45-ae8b-983a1cf249fa admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.096599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc22f8-561a-4e45-ae8b-983a1cf249fa admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.101653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f6628-50e8-40ee-b003-b4160ea27200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.102356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f6628-50e8-40ee-b003-b4160ea27200 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.102696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f6628-50e8-40ee-b003-b4160ea27200 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.103034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f6628-50e8-40ee-b003-b4160ea27200 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.103671 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f6628-50e8-40ee-b003-b4160ea27200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.105576 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f6628-50e8-40ee-b003-b4160ea27200 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.108557 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 796/1586] 10.222.0.55 () {58 vars in 1059 bytes} [Thu Sep 8 13:52:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:22.111579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbaa55-4779-4b5c-8244-fd6ec3b0ee5f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.123784 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbaa55-4779-4b5c-8244-fd6ec3b0ee5f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.128278 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 791/1587] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:22 2022] POST /identity/v3/users => generated 284 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:22.129098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f6628-50e8-40ee-b003-b4160ea27200 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.129864 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9159946-dc82-4229-8750-76f6c156e807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.130771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9159946-dc82-4229-8750-76f6c156e807 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.131080 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9159946-dc82-4229-8750-76f6c156e807 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.131381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9159946-dc82-4229-8750-76f6c156e807 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.132034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9159946-dc82-4229-8750-76f6c156e807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.133665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9159946-dc82-4229-8750-76f6c156e807 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.138827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f6628-50e8-40ee-b003-b4160ea27200 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.143970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6aa1e-bfa2-4a54-9fa4-945b4f28ec17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.144556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6aa1e-bfa2-4a54-9fa4-945b4f28ec17 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.144665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 797/1588] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:22.144837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6aa1e-bfa2-4a54-9fa4-945b4f28ec17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.145109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d6aa1e-bfa2-4a54-9fa4-945b4f28ec17 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.145446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9159946-dc82-4229-8750-76f6c156e807 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.145766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6aa1e-bfa2-4a54-9fa4-945b4f28ec17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.147352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d6aa1e-bfa2-4a54-9fa4-945b4f28ec17 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.155444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9159946-dc82-4229-8750-76f6c156e807 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.158470 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 792/1589] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:22.159175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6aa1e-bfa2-4a54-9fa4-945b4f28ec17 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.163127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c405e-df67-4843-a3d9-836ce7726fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.163726 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c405e-df67-4843-a3d9-836ce7726fff admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.164043 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c405e-df67-4843-a3d9-836ce7726fff admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.164320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c405e-df67-4843-a3d9-836ce7726fff admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.164970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c405e-df67-4843-a3d9-836ce7726fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.166573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c405e-df67-4843-a3d9-836ce7726fff admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.168619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d6aa1e-bfa2-4a54-9fa4-945b4f28ec17 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.173213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92750fa4-2ad0-409e-a4fa-e9c10bdee888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.173808 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92750fa4-2ad0-409e-a4fa-e9c10bdee888 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.174089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92750fa4-2ad0-409e-a4fa-e9c10bdee888 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.174357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92750fa4-2ad0-409e-a4fa-e9c10bdee888 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.174982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92750fa4-2ad0-409e-a4fa-e9c10bdee888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.176537 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92750fa4-2ad0-409e-a4fa-e9c10bdee888 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.176649 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 798/1590] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:22.178954 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c405e-df67-4843-a3d9-836ce7726fff admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.189097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 793/1591] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:22.189097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92750fa4-2ad0-409e-a4fa-e9c10bdee888 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.189339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c405e-df67-4843-a3d9-836ce7726fff admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.194216 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d6b01-0744-402e-8efc-99e8eff4bdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.194801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d6b01-0744-402e-8efc-99e8eff4bdef admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.195077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d6b01-0744-402e-8efc-99e8eff4bdef admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.195347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d6b01-0744-402e-8efc-99e8eff4bdef admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.195976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d6b01-0744-402e-8efc-99e8eff4bdef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.196397 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92750fa4-2ad0-409e-a4fa-e9c10bdee888 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.197553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d6b01-0744-402e-8efc-99e8eff4bdef admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.200881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777126d-10c5-43e7-827f-55c1b9014b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.201454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777126d-10c5-43e7-827f-55c1b9014b6e admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.201870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777126d-10c5-43e7-827f-55c1b9014b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.202019 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777126d-10c5-43e7-827f-55c1b9014b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.203174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777126d-10c5-43e7-827f-55c1b9014b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.204745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777126d-10c5-43e7-827f-55c1b9014b6e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.208496 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 799/1592] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:22.209281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d6b01-0744-402e-8efc-99e8eff4bdef admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.217232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 794/1593] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:22.218083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777126d-10c5-43e7-827f-55c1b9014b6e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.221541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d6b01-0744-402e-8efc-99e8eff4bdef admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.226992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2363277-5fce-427d-9e46-f6db68537a1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.227574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2363277-5fce-427d-9e46-f6db68537a1f service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.227702 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777126d-10c5-43e7-827f-55c1b9014b6e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.227854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2363277-5fce-427d-9e46-f6db68537a1f service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.228125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2363277-5fce-427d-9e46-f6db68537a1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.232201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91840c-33d9-48ae-857c-18d181be1a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.232770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91840c-33d9-48ae-857c-18d181be1a70 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.233052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91840c-33d9-48ae-857c-18d181be1a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.233328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91840c-33d9-48ae-857c-18d181be1a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.234498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91840c-33d9-48ae-857c-18d181be1a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.236064 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91840c-33d9-48ae-857c-18d181be1a70 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.247395 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 795/1594] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:22.248184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91840c-33d9-48ae-857c-18d181be1a70 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.257413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91840c-33d9-48ae-857c-18d181be1a70 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.261980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ce734-87d1-44c6-8eea-29881e8a4257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.262551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ce734-87d1-44c6-8eea-29881e8a4257 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.262827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ce734-87d1-44c6-8eea-29881e8a4257 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.263099 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ce734-87d1-44c6-8eea-29881e8a4257 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.263721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ce734-87d1-44c6-8eea-29881e8a4257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.265830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ce734-87d1-44c6-8eea-29881e8a4257 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.275723 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 796/1595] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:22.276330 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ce734-87d1-44c6-8eea-29881e8a4257 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.285643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ce734-87d1-44c6-8eea-29881e8a4257 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.287928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2363277-5fce-427d-9e46-f6db68537a1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.289504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2363277-5fce-427d-9e46-f6db68537a1f service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.290203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8647197-7350-46c3-889d-d0dd1c9e84f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.290771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8647197-7350-46c3-889d-d0dd1c9e84f6 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.291047 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8647197-7350-46c3-889d-d0dd1c9e84f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.291315 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8647197-7350-46c3-889d-d0dd1c9e84f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.291933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8647197-7350-46c3-889d-d0dd1c9e84f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.293496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8647197-7350-46c3-889d-d0dd1c9e84f6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.299238 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 800/1596] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:22.300050 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2363277-5fce-427d-9e46-f6db68537a1f service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.306820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 797/1597] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:22.306820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8647197-7350-46c3-889d-d0dd1c9e84f6 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.312966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2363277-5fce-427d-9e46-f6db68537a1f service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.314503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8647197-7350-46c3-889d-d0dd1c9e84f6 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.317606 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881caba-d0ab-41db-af03-f02c3ae30e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.318277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881caba-d0ab-41db-af03-f02c3ae30e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.318550 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881caba-d0ab-41db-af03-f02c3ae30e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.318819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881caba-d0ab-41db-af03-f02c3ae30e3a admin admin] PATH_INFO: `/v3/projects/98c2fd35f46c4314874bcd6c09c08d4f/users/e13c9bee1a1c4a6b8a215cfba256c51e/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.319069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a98017-986b-439a-a931-33e75297cf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.319639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a98017-986b-439a-a931-33e75297cf7f admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.319915 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a98017-986b-439a-a931-33e75297cf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.320183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a98017-986b-439a-a931-33e75297cf7f admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.320806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a98017-986b-439a-a931-33e75297cf7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.322410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a98017-986b-439a-a931-33e75297cf7f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.336058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 798/1598] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:22.336058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a98017-986b-439a-a931-33e75297cf7f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.337785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881caba-d0ab-41db-af03-f02c3ae30e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=98c2fd35f46c4314874bcd6c09c08d4f, user_id=e13c9bee1a1c4a6b8a215cfba256c51e, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.341052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881caba-d0ab-41db-af03-f02c3ae30e3a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.346481 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a98017-986b-439a-a931-33e75297cf7f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.353997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8516a-7739-46c8-afd2-a0b8c197d1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.354990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 801/1599] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:22 2022] PUT /identity/v3/projects/98c2fd35f46c4314874bcd6c09c08d4f/users/e13c9bee1a1c4a6b8a215cfba256c51e/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:22.355247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8516a-7739-46c8-afd2-a0b8c197d1fd admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.355663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8516a-7739-46c8-afd2-a0b8c197d1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.355845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881caba-d0ab-41db-af03-f02c3ae30e3a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.356298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8516a-7739-46c8-afd2-a0b8c197d1fd admin admin] PATH_INFO: `/v3/projects/b868b7f8289047369b2fcbb500f353e5/users/55d8e06905644ac39777328db72526ca/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.374103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8516a-7739-46c8-afd2-a0b8c197d1fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b868b7f8289047369b2fcbb500f353e5, user_id=55d8e06905644ac39777328db72526ca, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.377030 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf8516a-7739-46c8-afd2-a0b8c197d1fd admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.392077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 799/1600] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:22 2022] PUT /identity/v3/projects/b868b7f8289047369b2fcbb500f353e5/users/55d8e06905644ac39777328db72526ca/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:22.392839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8516a-7739-46c8-afd2-a0b8c197d1fd admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.414804 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:22.427583 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881caba-d0ab-41db-af03-f02c3ae30e3a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.452869 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:22.465678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf8516a-7739-46c8-afd2-a0b8c197d1fd admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.471721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ee35a-1c60-4d87-bef1-97f52824cf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.472339 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ee35a-1c60-4d87-bef1-97f52824cf59 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.472621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ee35a-1c60-4d87-bef1-97f52824cf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.472924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ee35a-1c60-4d87-bef1-97f52824cf59 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.473997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ee35a-1c60-4d87-bef1-97f52824cf59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.475545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ee35a-1c60-4d87-bef1-97f52824cf59 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.485630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 800/1601] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:22.486439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ee35a-1c60-4d87-bef1-97f52824cf59 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.493136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a983e-71f5-4317-a1b3-220a48feab4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.494077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a983e-71f5-4317-a1b3-220a48feab4d admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.494357 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a983e-71f5-4317-a1b3-220a48feab4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.494631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a983e-71f5-4317-a1b3-220a48feab4d admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.495865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a983e-71f5-4317-a1b3-220a48feab4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.497483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a983e-71f5-4317-a1b3-220a48feab4d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.498764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ee35a-1c60-4d87-bef1-97f52824cf59 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.505082 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e07ca-13dd-40a0-bebc-45543de9a294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.506063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e07ca-13dd-40a0-bebc-45543de9a294 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.506345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e07ca-13dd-40a0-bebc-45543de9a294 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.506619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e07ca-13dd-40a0-bebc-45543de9a294 admin admin] PATH_INFO: `/v3/projects/127bce2080d54fe4871b5aaa6cc394b5/users/9f07158e4fd1430e839a1592e1e11b58/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.508655 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 802/1602] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:22.512600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a983e-71f5-4317-a1b3-220a48feab4d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.523575 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a983e-71f5-4317-a1b3-220a48feab4d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.527057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e07ca-13dd-40a0-bebc-45543de9a294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=127bce2080d54fe4871b5aaa6cc394b5, user_id=9f07158e4fd1430e839a1592e1e11b58, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.529563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea4850-6172-46e0-b89e-3f970a2ddd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.529977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e07ca-13dd-40a0-bebc-45543de9a294 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.531213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea4850-6172-46e0-b89e-3f970a2ddd4e admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.531700 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea4850-6172-46e0-b89e-3f970a2ddd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.532173 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea4850-6172-46e0-b89e-3f970a2ddd4e admin admin] PATH_INFO: `/v3/projects/b4ca185497284e9780eba8fc2e713d6e/users/2e48a8c1b91142cf9d1af17f28e41961/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.544978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 801/1603] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:22 2022] PUT /identity/v3/projects/127bce2080d54fe4871b5aaa6cc394b5/users/9f07158e4fd1430e839a1592e1e11b58/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:22.545774 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e07ca-13dd-40a0-bebc-45543de9a294 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.551763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea4850-6172-46e0-b89e-3f970a2ddd4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ca185497284e9780eba8fc2e713d6e, user_id=2e48a8c1b91142cf9d1af17f28e41961, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.554479 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea4850-6172-46e0-b89e-3f970a2ddd4e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.569824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 803/1604] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:22 2022] PUT /identity/v3/projects/b4ca185497284e9780eba8fc2e713d6e/users/2e48a8c1b91142cf9d1af17f28e41961/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:22.573562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cfd59-d294-45e5-98c5-0e79fa4cf11e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.573562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cfd59-d294-45e5-98c5-0e79fa4cf11e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.573562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cfd59-d294-45e5-98c5-0e79fa4cf11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.590523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ff4cfd59-d294-45e5-98c5-0e79fa4cf11e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:22.591619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ff4cfd59-d294-45e5-98c5-0e79fa4cf11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:22.611860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e07ca-13dd-40a0-bebc-45543de9a294 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.655656 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 804/1605] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:22.656908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840faf25-17e1-4a72-baff-46c070e45ec5 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.657197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840faf25-17e1-4a72-baff-46c070e45ec5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.657474 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840faf25-17e1-4a72-baff-46c070e45ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.672486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73537b-2535-4ff2-a5c1-f14a508b9461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.673275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73537b-2535-4ff2-a5c1-f14a508b9461 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.673654 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73537b-2535-4ff2-a5c1-f14a508b9461 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.674066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73537b-2535-4ff2-a5c1-f14a508b9461 admin admin] PATH_INFO: `/v3/projects/b2b7fd2f4a4e478bb57d996da77bb2cd/users/b7cd12278c4c456f8e33bc43d6578cb6/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.675726 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-840faf25-17e1-4a72-baff-46c070e45ec5 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:22.676924 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-840faf25-17e1-4a72-baff-46c070e45ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:22.684561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 805/1606] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:22.685314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840faf25-17e1-4a72-baff-46c070e45ec5 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.691937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73537b-2535-4ff2-a5c1-f14a508b9461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b7fd2f4a4e478bb57d996da77bb2cd, user_id=b7cd12278c4c456f8e33bc43d6578cb6, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.695133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73537b-2535-4ff2-a5c1-f14a508b9461 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.695271 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840faf25-17e1-4a72-baff-46c070e45ec5 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.699815 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c889b9-f546-4759-b369-2154da971cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.700590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c889b9-f546-4759-b369-2154da971cca admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.700860 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c889b9-f546-4759-b369-2154da971cca admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.701167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c889b9-f546-4759-b369-2154da971cca admin admin] PATH_INFO: `/v3/projects/18ab1673e1f84248a53ff27b3ccf0257/users/440e29b63082400a93427fdd449e4a97/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.708556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 802/1607] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:22 2022] PUT /identity/v3/projects/b2b7fd2f4a4e478bb57d996da77bb2cd/users/b7cd12278c4c456f8e33bc43d6578cb6/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:22.715102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73537b-2535-4ff2-a5c1-f14a508b9461 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.717476 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c889b9-f546-4759-b369-2154da971cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=18ab1673e1f84248a53ff27b3ccf0257, user_id=440e29b63082400a93427fdd449e4a97, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.720200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c889b9-f546-4759-b369-2154da971cca admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.732636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 806/1608] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:22 2022] PUT /identity/v3/projects/18ab1673e1f84248a53ff27b3ccf0257/users/440e29b63082400a93427fdd449e4a97/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:22.744031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e30f78-1393-45c1-b1eb-5ca8170726e1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.744031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e30f78-1393-45c1-b1eb-5ca8170726e1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.744031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e30f78-1393-45c1-b1eb-5ca8170726e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.745355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:22.757312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b1e30f78-1393-45c1-b1eb-5ca8170726e1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:22.758558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b1e30f78-1393-45c1-b1eb-5ca8170726e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:22.781133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73537b-2535-4ff2-a5c1-f14a508b9461 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.822560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 807/1609] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:22.823824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0660fc-1cb1-4d72-97c7-d8bd7a4cdaa0 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.824108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0660fc-1cb1-4d72-97c7-d8bd7a4cdaa0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.824384 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0660fc-1cb1-4d72-97c7-d8bd7a4cdaa0 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.825783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 808/1610] 10.222.0.55 () {54 vars in 912 bytes} [Thu Sep 8 13:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:22.826652 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0070a2-34da-4aa5-bb09-cd13f70cb0e5 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.826934 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0070a2-34da-4aa5-bb09-cd13f70cb0e5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.827204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0070a2-34da-4aa5-bb09-cd13f70cb0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.844379 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3d21a-0978-4e91-8229-bc325bf42175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.844956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3d21a-0978-4e91-8229-bc325bf42175 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.845237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3d21a-0978-4e91-8229-bc325bf42175 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.845505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3d21a-0978-4e91-8229-bc325bf42175 admin admin] PATH_INFO: `/v3/projects/6998e0c9a62e4f54a27faf56dbeb543a/users/96744eb6a9cf4895b6101e209d1fdd29/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.846832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-dd0070a2-34da-4aa5-bb09-cd13f70cb0e5 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:22.848110 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dd0070a2-34da-4aa5-bb09-cd13f70cb0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:22.855708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 809/1611] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:22.856482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0070a2-34da-4aa5-bb09-cd13f70cb0e5 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.861361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3d21a-0978-4e91-8229-bc325bf42175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6998e0c9a62e4f54a27faf56dbeb543a, user_id=96744eb6a9cf4895b6101e209d1fdd29, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:22.862448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:22.864887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3d21a-0978-4e91-8229-bc325bf42175 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:22.878784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 803/1612] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:22 2022] PUT /identity/v3/projects/6998e0c9a62e4f54a27faf56dbeb543a/users/96744eb6a9cf4895b6101e209d1fdd29/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:22.882618 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3d21a-0978-4e91-8229-bc325bf42175 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:22.884398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a1e3d21a-0978-4e91-8229-bc325bf42175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:22.914305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0070a2-34da-4aa5-bb09-cd13f70cb0e5 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.944499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3d21a-0978-4e91-8229-bc325bf42175 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:22.949033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8833b72-3b53-469f-9763-383320d9135c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.949668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833b72-3b53-469f-9763-383320d9135c service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.949968 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833b72-3b53-469f-9763-383320d9135c service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.950239 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8833b72-3b53-469f-9763-383320d9135c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:22.964847 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d423c-b506-47b0-985b-8b00ca14e74b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:22.965453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d423c-b506-47b0-985b-8b00ca14e74b service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:22.965757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d423c-b506-47b0-985b-8b00ca14e74b service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:22.966028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d423c-b506-47b0-985b-8b00ca14e74b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.014211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8833b72-3b53-469f-9763-383320d9135c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:23.016134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8833b72-3b53-469f-9763-383320d9135c service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:23.021069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:23.026648 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 804/1613] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:23.028523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8e99d-8905-4942-b59d-89fcfcd0592d None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.028818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8e99d-8905-4942-b59d-89fcfcd0592d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.029437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8e99d-8905-4942-b59d-89fcfcd0592d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.034557 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d423c-b506-47b0-985b-8b00ca14e74b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:23.036097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d423c-b506-47b0-985b-8b00ca14e74b service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:23.048261 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 810/1614] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:23.049698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81088ab9-73b6-47c2-a3c4-1c880008510c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.050036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81088ab9-73b6-47c2-a3c4-1c880008510c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.050353 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81088ab9-73b6-47c2-a3c4-1c880008510c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.062487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-6da8e99d-8905-4942-b59d-89fcfcd0592d None None] MFA Rules not processed for user `b7cd12278c4c456f8e33bc43d6578cb6`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:23.063720 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-6da8e99d-8905-4942-b59d-89fcfcd0592d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:23.094291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-81088ab9-73b6-47c2-a3c4-1c880008510c None None] MFA Rules not processed for user `440e29b63082400a93427fdd449e4a97`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:23.096120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-81088ab9-73b6-47c2-a3c4-1c880008510c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:23.149895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 805/1615] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:23.152050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e86a4-24e9-4deb-82c7-fae67cc06442 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.152342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e86a4-24e9-4deb-82c7-fae67cc06442 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.152920 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577e86a4-24e9-4deb-82c7-fae67cc06442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.180257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-577e86a4-24e9-4deb-82c7-fae67cc06442 None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:23.181877 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-577e86a4-24e9-4deb-82c7-fae67cc06442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:23.195574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:23.209601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 811/1616] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:23.211484 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81088ab9-73b6-47c2-a3c4-1c880008510c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:23.224056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81088ab9-73b6-47c2-a3c4-1c880008510c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:23.228311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef4bfb-fd5e-4d2f-b75c-1739c18e1b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:23.229125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef4bfb-fd5e-4d2f-b75c-1739c18e1b79 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.229473 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef4bfb-fd5e-4d2f-b75c-1739c18e1b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.230250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef4bfb-fd5e-4d2f-b75c-1739c18e1b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.242520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef4bfb-fd5e-4d2f-b75c-1739c18e1b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:23.244920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef4bfb-fd5e-4d2f-b75c-1739c18e1b79 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:23.257884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 812/1617] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:23.258594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef4bfb-fd5e-4d2f-b75c-1739c18e1b79 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:23.267190 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef4bfb-fd5e-4d2f-b75c-1739c18e1b79 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:23.271483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f5847-1462-4484-9881-683697456d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:23.272101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f5847-1462-4484-9881-683697456d94 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.272448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f5847-1462-4484-9881-683697456d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.272788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f5847-1462-4484-9881-683697456d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.284484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 806/1618] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 13:52:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:23.287414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483261e4-bbf6-4966-85d3-01c739df539f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.287781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f5847-1462-4484-9881-683697456d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:23.288190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483261e4-bbf6-4966-85d3-01c739df539f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.288666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483261e4-bbf6-4966-85d3-01c739df539f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.289572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f5847-1462-4484-9881-683697456d94 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:23.310957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 813/1619] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:23.312957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aede723-d759-484f-a7e6-5159252cdcd2 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.313349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aede723-d759-484f-a7e6-5159252cdcd2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.313721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aede723-d759-484f-a7e6-5159252cdcd2 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.315195 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 814/1620] 10.222.0.55 () {54 vars in 962 bytes} [Thu Sep 8 13:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:23.319341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-483261e4-bbf6-4966-85d3-01c739df539f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:23.320766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aede723-d759-484f-a7e6-5159252cdcd2 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:23.321558 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-483261e4-bbf6-4966-85d3-01c739df539f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:23.333037 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 807/1621] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:23.334337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483261e4-bbf6-4966-85d3-01c739df539f None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:23.346270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aede723-d759-484f-a7e6-5159252cdcd2 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:23.352253 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872417e5-cdad-4828-ada3-f94031adc5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:23.353139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872417e5-cdad-4828-ada3-f94031adc5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.353559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872417e5-cdad-4828-ada3-f94031adc5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.354156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872417e5-cdad-4828-ada3-f94031adc5a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.355080 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872417e5-cdad-4828-ada3-f94031adc5a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:23.357680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872417e5-cdad-4828-ada3-f94031adc5a1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:23.407594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 815/1622] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:23 2022] POST /identity/v3/projects => generated 365 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:23.409905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79366ea-bbb1-435a-b29e-19937cce8856 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.410477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79366ea-bbb1-435a-b29e-19937cce8856 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.411103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79366ea-bbb1-435a-b29e-19937cce8856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.412938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483261e4-bbf6-4966-85d3-01c739df539f None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:23.440525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b79366ea-bbb1-435a-b29e-19937cce8856 None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:23.443013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b79366ea-bbb1-435a-b29e-19937cce8856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:23.466395 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:23.496462 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbe641-e11e-449a-b00d-c80308366c6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:23.497359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbe641-e11e-449a-b00d-c80308366c6b service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.499822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbe641-e11e-449a-b00d-c80308366c6b service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.499822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbe641-e11e-449a-b00d-c80308366c6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.596870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 816/1623] 10.222.0.55 () {58 vars in 1059 bytes} [Thu Sep 8 13:52:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:23.601503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79366ea-bbb1-435a-b29e-19937cce8856 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:23.628103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afbe641-e11e-449a-b00d-c80308366c6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:23.629668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afbe641-e11e-449a-b00d-c80308366c6b service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:23.645669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 808/1624] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:23.646585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbe641-e11e-449a-b00d-c80308366c6b service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:23.782992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79366ea-bbb1-435a-b29e-19937cce8856 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:23.797602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4f265-d25b-412c-9979-bd035c05921c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:23.798419 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4f265-d25b-412c-9979-bd035c05921c service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.798863 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4f265-d25b-412c-9979-bd035c05921c service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.799152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4f265-d25b-412c-9979-bd035c05921c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.802402 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbe641-e11e-449a-b00d-c80308366c6b service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:23.816665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd15737-4e3f-40d0-9b9a-548987c5617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:23.818374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd15737-4e3f-40d0-9b9a-548987c5617d admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.818666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd15737-4e3f-40d0-9b9a-548987c5617d admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.820079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd15737-4e3f-40d0-9b9a-548987c5617d admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.821082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4f265-d25b-412c-9979-bd035c05921c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:23.822255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd15737-4e3f-40d0-9b9a-548987c5617d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:23.823307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4f265-d25b-412c-9979-bd035c05921c service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:23.827393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd15737-4e3f-40d0-9b9a-548987c5617d admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:23.849511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 817/1625] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:23.850550 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4f265-d25b-412c-9979-bd035c05921c service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:23.879056 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 809/1626] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:23 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:23.883758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd15737-4e3f-40d0-9b9a-548987c5617d admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:23.885686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4f265-d25b-412c-9979-bd035c05921c service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:23.892347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08927e-4211-4b5e-96ad-0b7a9d89a9cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:23.893137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08927e-4211-4b5e-96ad-0b7a9d89a9cf service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.893518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08927e-4211-4b5e-96ad-0b7a9d89a9cf service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.894449 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08927e-4211-4b5e-96ad-0b7a9d89a9cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.911856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd15737-4e3f-40d0-9b9a-548987c5617d admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:23.925826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b8249-bcc6-4014-8b16-f8390e6bc29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:23.925826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b8249-bcc6-4014-8b16-f8390e6bc29e admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.925826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b8249-bcc6-4014-8b16-f8390e6bc29e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.931360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b8249-bcc6-4014-8b16-f8390e6bc29e admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:23.932241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b8249-bcc6-4014-8b16-f8390e6bc29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:23.933999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b8249-bcc6-4014-8b16-f8390e6bc29e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:23.950645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 810/1627] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:23.954368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b8249-bcc6-4014-8b16-f8390e6bc29e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:23.970052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:23.984520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b8249-bcc6-4014-8b16-f8390e6bc29e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:23.990788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5fb181-c6d9-4a02-8352-905c72d7c8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:23.991798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fb181-c6d9-4a02-8352-905c72d7c8de admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:23.992267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fb181-c6d9-4a02-8352-905c72d7c8de admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:23.992718 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5fb181-c6d9-4a02-8352-905c72d7c8de admin admin] PATH_INFO: `/v3/projects/1a35514109a1451a81faa1e8eeee4362/users/3bf1a6a52945427c84d2364d29350a21/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:24.004249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08927e-4211-4b5e-96ad-0b7a9d89a9cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:24.006143 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08927e-4211-4b5e-96ad-0b7a9d89a9cf service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:24.018013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5fb181-c6d9-4a02-8352-905c72d7c8de admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a35514109a1451a81faa1e8eeee4362, user_id=3bf1a6a52945427c84d2364d29350a21, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:24.023616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5fb181-c6d9-4a02-8352-905c72d7c8de admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:24.026874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 818/1628] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:24.044756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 811/1629] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:23 2022] PUT /identity/v3/projects/1a35514109a1451a81faa1e8eeee4362/users/3bf1a6a52945427c84d2364d29350a21/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:24.053103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08927e-4211-4b5e-96ad-0b7a9d89a9cf service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:24.086975 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08927e-4211-4b5e-96ad-0b7a9d89a9cf service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:24.216931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23249a65-fd3f-4dc5-b678-6223441ce0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:24.217591 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23249a65-fd3f-4dc5-b678-6223441ce0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:24.217911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23249a65-fd3f-4dc5-b678-6223441ce0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:24.218758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23249a65-fd3f-4dc5-b678-6223441ce0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:24.219413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23249a65-fd3f-4dc5-b678-6223441ce0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:24.221028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23249a65-fd3f-4dc5-b678-6223441ce0b8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:24.249713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 819/1630] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:24.251129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5fb181-c6d9-4a02-8352-905c72d7c8de admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:24.270579 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5fb181-c6d9-4a02-8352-905c72d7c8de admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:24.283310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2aed1b-d335-4249-8c86-ed179501de16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:24.285537 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aed1b-d335-4249-8c86-ed179501de16 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:24.285699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aed1b-d335-4249-8c86-ed179501de16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:24.286519 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aed1b-d335-4249-8c86-ed179501de16 admin admin] PATH_INFO: `/v3/projects/1a35514109a1451a81faa1e8eeee4362/users/3bf1a6a52945427c84d2364d29350a21/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:24.311896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2aed1b-d335-4249-8c86-ed179501de16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a35514109a1451a81faa1e8eeee4362, user_id=3bf1a6a52945427c84d2364d29350a21, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:24.321939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2aed1b-d335-4249-8c86-ed179501de16 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:24.357778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 812/1631] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:24 2022] PUT /identity/v3/projects/1a35514109a1451a81faa1e8eeee4362/users/3bf1a6a52945427c84d2364d29350a21/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:52:24.367706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f00798-cb43-4f67-90a4-b80031c08394 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:24.368040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f00798-cb43-4f67-90a4-b80031c08394 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:24.368259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f00798-cb43-4f67-90a4-b80031c08394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:24.425207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-43f00798-cb43-4f67-90a4-b80031c08394 None None] MFA Rules not processed for user `3bf1a6a52945427c84d2364d29350a21`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:24.426523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-43f00798-cb43-4f67-90a4-b80031c08394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:24.448896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7103bb-8ad9-4d08-8171-968c54069f39 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:24.449205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7103bb-8ad9-4d08-8171-968c54069f39 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:24.450055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7103bb-8ad9-4d08-8171-968c54069f39 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:24.451675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 813/1632] 10.222.0.55 () {54 vars in 912 bytes} [Thu Sep 8 13:52:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:24.467906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5b42f-13de-4ea9-87e1-44ef33bd677b None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:24.469108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5b42f-13de-4ea9-87e1-44ef33bd677b None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:24.469634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5b42f-13de-4ea9-87e1-44ef33bd677b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:24.506712 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-21a5b42f-13de-4ea9-87e1-44ef33bd677b None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:24.510035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-21a5b42f-13de-4ea9-87e1-44ef33bd677b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:24.536138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:24.572486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 820/1633] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:24.581547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f00798-cb43-4f67-90a4-b80031c08394 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:24.708995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 814/1634] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 13:52:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 08 13:52:24.718538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f00798-cb43-4f67-90a4-b80031c08394 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:24.728910 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2b235-6449-4121-958a-741cfb61f9eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:24.730805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2b235-6449-4121-958a-741cfb61f9eb service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:24.731141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2b235-6449-4121-958a-741cfb61f9eb service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:24.732741 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2b235-6449-4121-958a-741cfb61f9eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:24.759627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2b235-6449-4121-958a-741cfb61f9eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:24.762062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2b235-6449-4121-958a-741cfb61f9eb service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:24.810198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 821/1635] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:24.821529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5b42f-13de-4ea9-87e1-44ef33bd677b None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:24.837580 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5b42f-13de-4ea9-87e1-44ef33bd677b None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:24.845888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd7a1c-f5fb-45bc-8fe0-d5a69d13af30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:24.850053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd7a1c-f5fb-45bc-8fe0-d5a69d13af30 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:24.851584 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd7a1c-f5fb-45bc-8fe0-d5a69d13af30 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:24.851584 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd7a1c-f5fb-45bc-8fe0-d5a69d13af30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:24.900730 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd7a1c-f5fb-45bc-8fe0-d5a69d13af30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:24.902251 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd7a1c-f5fb-45bc-8fe0-d5a69d13af30 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:24.925700 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 815/1636] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:25.261392 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b00e25-987c-41c2-af0c-6bdc182bb34f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:25.261681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b00e25-987c-41c2-af0c-6bdc182bb34f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:25.262577 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b00e25-987c-41c2-af0c-6bdc182bb34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:25.286009 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:25.326291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-06b00e25-987c-41c2-af0c-6bdc182bb34f None None] MFA Rules not processed for user `e13c9bee1a1c4a6b8a215cfba256c51e`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:25.329682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-06b00e25-987c-41c2-af0c-6bdc182bb34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:25.458124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd7a1c-f5fb-45bc-8fe0-d5a69d13af30 service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:25.460492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 822/1637] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:25 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:25.591708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd7a1c-f5fb-45bc-8fe0-d5a69d13af30 service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:25.607843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbdb4a-fab1-47ac-8dd6-fb247bf7fc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:25.608817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbdb4a-fab1-47ac-8dd6-fb247bf7fc81 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:25.610209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbdb4a-fab1-47ac-8dd6-fb247bf7fc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:25.610887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbdb4a-fab1-47ac-8dd6-fb247bf7fc81 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:25.611955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbdb4a-fab1-47ac-8dd6-fb247bf7fc81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:25.617521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbdb4a-fab1-47ac-8dd6-fb247bf7fc81 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:25.693421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b00e25-987c-41c2-af0c-6bdc182bb34f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:25.703277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 816/1638] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:25 2022] POST /identity/v3/projects => generated 403 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:25.723807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b00e25-987c-41c2-af0c-6bdc182bb34f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:25.876191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b97498-b41d-47e8-99e3-952f2de79e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:25.876792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b97498-b41d-47e8-99e3-952f2de79e37 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:25.877708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b97498-b41d-47e8-99e3-952f2de79e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:25.877708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b97498-b41d-47e8-99e3-952f2de79e37 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:25.878558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b97498-b41d-47e8-99e3-952f2de79e37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:25.881487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b97498-b41d-47e8-99e3-952f2de79e37 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:25.924213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 823/1639] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:25 2022] POST /identity/v3/users => generated 308 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:25.926084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbdb4a-fab1-47ac-8dd6-fb247bf7fc81 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:25.942274 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbdb4a-fab1-47ac-8dd6-fb247bf7fc81 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:25.953805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129e880-8036-4607-9278-13eff18379ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:25.955737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129e880-8036-4607-9278-13eff18379ef admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:25.956024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129e880-8036-4607-9278-13eff18379ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:25.956299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129e880-8036-4607-9278-13eff18379ef admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:25.958209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129e880-8036-4607-9278-13eff18379ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:25.964805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129e880-8036-4607-9278-13eff18379ef admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:25.993773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 817/1640] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:52:25.995750 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b97498-b41d-47e8-99e3-952f2de79e37 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:26.011766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b97498-b41d-47e8-99e3-952f2de79e37 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:26.024670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e9616-c34a-4dbb-8e0c-9853eea37824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:26.026475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e9616-c34a-4dbb-8e0c-9853eea37824 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:26.027030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e9616-c34a-4dbb-8e0c-9853eea37824 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:26.028556 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e9616-c34a-4dbb-8e0c-9853eea37824 admin admin] PATH_INFO: `/v3/projects/8d113649d43f48c499dd573c9880c5de/users/26c6b72729a24bf4831b2964524af902/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:26.070293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e9616-c34a-4dbb-8e0c-9853eea37824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d113649d43f48c499dd573c9880c5de, user_id=26c6b72729a24bf4831b2964524af902, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:26.074150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e9616-c34a-4dbb-8e0c-9853eea37824 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:26.118787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 824/1641] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:25 2022] PUT /identity/v3/projects/8d113649d43f48c499dd573c9880c5de/users/26c6b72729a24bf4831b2964524af902/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:26.123004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129e880-8036-4607-9278-13eff18379ef admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:26.138664 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129e880-8036-4607-9278-13eff18379ef admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:26.303831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888302f9-6bb3-4277-baca-562c8beba035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:26.304455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888302f9-6bb3-4277-baca-562c8beba035 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:26.304737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888302f9-6bb3-4277-baca-562c8beba035 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:26.305011 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888302f9-6bb3-4277-baca-562c8beba035 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:26.305658 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888302f9-6bb3-4277-baca-562c8beba035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:26.307307 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888302f9-6bb3-4277-baca-562c8beba035 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:26.335605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 818/1642] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:26.339519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e9616-c34a-4dbb-8e0c-9853eea37824 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:26.353573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e9616-c34a-4dbb-8e0c-9853eea37824 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:26.366504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace75c0a-5644-4e74-bae0-3710827ac0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:26.367252 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace75c0a-5644-4e74-bae0-3710827ac0c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:26.367539 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace75c0a-5644-4e74-bae0-3710827ac0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:26.367818 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace75c0a-5644-4e74-bae0-3710827ac0c8 admin admin] PATH_INFO: `/v3/projects/8d113649d43f48c499dd573c9880c5de/users/26c6b72729a24bf4831b2964524af902/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:26.373162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace75c0a-5644-4e74-bae0-3710827ac0c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d113649d43f48c499dd573c9880c5de, user_id=26c6b72729a24bf4831b2964524af902, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:26.378617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace75c0a-5644-4e74-bae0-3710827ac0c8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:26.410326 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 825/1643] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:26 2022] PUT /identity/v3/projects/8d113649d43f48c499dd573c9880c5de/users/26c6b72729a24bf4831b2964524af902/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:27.688062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03143da-fe62-4730-a0fa-6c449ceaaf6f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:27.688539 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03143da-fe62-4730-a0fa-6c449ceaaf6f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:27.688631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03143da-fe62-4730-a0fa-6c449ceaaf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:27.703763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:27.776300 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-d03143da-fe62-4730-a0fa-6c449ceaaf6f None None] MFA Rules not processed for user `26c6b72729a24bf4831b2964524af902`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:27.779809 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d03143da-fe62-4730-a0fa-6c449ceaaf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:27.940272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace75c0a-5644-4e74-bae0-3710827ac0c8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:28.013305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 819/1644] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:27 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:28.029578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:28.062659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace75c0a-5644-4e74-bae0-3710827ac0c8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:28.074777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:28.075693 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:28.076182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:28.077697 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] PATH_INFO: `/v3/users/440e29b63082400a93427fdd449e4a97` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:28.079702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] RBAC: Authorizing `identity:delete_user(user_id=440e29b63082400a93427fdd449e4a97)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:28.082233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:28.213299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '440e29b63082400a93427fdd449e4a97'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:28.229163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '440e29b63082400a93427fdd449e4a97'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:28.230541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '440e29b63082400a93427fdd449e4a97'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:28.231426 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '440e29b63082400a93427fdd449e4a97'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:28.247446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '440e29b63082400a93427fdd449e4a97'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:28.303539 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7fa8a-02e8-4c83-b864-c14e50bb580d None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:28.303832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7fa8a-02e8-4c83-b864-c14e50bb580d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:28.304109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7fa8a-02e8-4c83-b864-c14e50bb580d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:28.306744 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 826/1645] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:52:27 2022] DELETE /identity/v3/users/440e29b63082400a93427fdd449e4a97 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:28.337450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-d9e7fa8a-02e8-4c83-b864-c14e50bb580d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:28.338758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d9e7fa8a-02e8-4c83-b864-c14e50bb580d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:28.418862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 820/1646] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:52:28.542824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:28.545706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:28.653769 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a0c49-219e-4b9d-88b4-d953dab5871f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:28.656636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315bd7a-a539-41d4-a0fb-986d1074349d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:28.657399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315bd7a-a539-41d4-a0fb-986d1074349d service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:28.657531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315bd7a-a539-41d4-a0fb-986d1074349d service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:28.657806 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315bd7a-a539-41d4-a0fb-986d1074349d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:28.670178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315bd7a-a539-41d4-a0fb-986d1074349d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:28.674592 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315bd7a-a539-41d4-a0fb-986d1074349d service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:28.688400 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 827/1647] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:30.169210 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0f9e2-3c4f-450d-a04d-c6108f376dd4 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:30.171861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0f9e2-3c4f-450d-a04d-c6108f376dd4 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:30.171861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0f9e2-3c4f-450d-a04d-c6108f376dd4 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:30.173833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 821/1648] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 13:52:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:30.206530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aac4ea-0a22-4635-ae4b-ba8f439f6b93 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:30.206999 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aac4ea-0a22-4635-ae4b-ba8f439f6b93 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:30.208101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aac4ea-0a22-4635-ae4b-ba8f439f6b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:30.255416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a5aac4ea-0a22-4635-ae4b-ba8f439f6b93 None None] MFA Rules not processed for user `e42fb05dbae64c758098c591d6ffa790`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:30.259543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a5aac4ea-0a22-4635-ae4b-ba8f439f6b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:30.436379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 828/1649] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 13:52:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:30.482489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0f9e2-3c4f-450d-a04d-c6108f376dd4 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:30.484540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-86a0f9e2-3c4f-450d-a04d-c6108f376dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:30.529221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0f9e2-3c4f-450d-a04d-c6108f376dd4 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:30.550934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c7941-19c6-41c1-8fe8-a492290521b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:30.551572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c7941-19c6-41c1-8fe8-a492290521b9 service placement] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:30.551856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c7941-19c6-41c1-8fe8-a492290521b9 service placement] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:30.552132 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c7941-19c6-41c1-8fe8-a492290521b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:30.578456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c7941-19c6-41c1-8fe8-a492290521b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:30.590997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c7941-19c6-41c1-8fe8-a492290521b9 service placement] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:30.618654 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 822/1650] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:30.754897 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aac4ea-0a22-4635-ae4b-ba8f439f6b93 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:30.756997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a5aac4ea-0a22-4635-ae4b-ba8f439f6b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:30.785462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aac4ea-0a22-4635-ae4b-ba8f439f6b93 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:30.797930 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:30.798617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:30.798899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:30.799387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] PATH_INFO: `/v3/users/b7cd12278c4c456f8e33bc43d6578cb6` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:30.804175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7cd12278c4c456f8e33bc43d6578cb6)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:30.812634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:30.816836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:30.936607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7cd12278c4c456f8e33bc43d6578cb6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:30.951713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7cd12278c4c456f8e33bc43d6578cb6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:30.953874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7cd12278c4c456f8e33bc43d6578cb6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:30.954737 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7cd12278c4c456f8e33bc43d6578cb6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:30.971402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7cd12278c4c456f8e33bc43d6578cb6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:31.023472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570518b-2179-4c4f-9ad3-f4a2a0111385 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:31.023756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570518b-2179-4c4f-9ad3-f4a2a0111385 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:31.024204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570518b-2179-4c4f-9ad3-f4a2a0111385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:31.035431 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 829/1651] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:52:30 2022] DELETE /identity/v3/users/b7cd12278c4c456f8e33bc43d6578cb6 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:31.064474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-0570518b-2179-4c4f-9ad3-f4a2a0111385 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:31.065940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0570518b-2179-4c4f-9ad3-f4a2a0111385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:31.180114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 823/1652] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:52:31.197321 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:31.200541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:31.377969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2aeb8-7192-43ff-8c71-de939acc4628 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:31.383535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087eec46-c4c9-4eaa-8213-3f8adc6da52c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:31.384484 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087eec46-c4c9-4eaa-8213-3f8adc6da52c service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:31.384966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087eec46-c4c9-4eaa-8213-3f8adc6da52c service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:31.385448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087eec46-c4c9-4eaa-8213-3f8adc6da52c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:31.400087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087eec46-c4c9-4eaa-8213-3f8adc6da52c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:31.401687 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087eec46-c4c9-4eaa-8213-3f8adc6da52c service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:31.433343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 830/1653] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:31.652667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570518b-2179-4c4f-9ad3-f4a2a0111385 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:31.654116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0570518b-2179-4c4f-9ad3-f4a2a0111385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:31.671724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570518b-2179-4c4f-9ad3-f4a2a0111385 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:31.688213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:31.689106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:31.689614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:31.690144 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] PATH_INFO: `/v3/projects/18ab1673e1f84248a53ff27b3ccf0257` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:31.695018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18ab1673e1f84248a53ff27b3ccf0257)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:31.699451 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:31.735202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:31.887966 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18ab1673e1f84248a53ff27b3ccf0257', 'request_id': 'req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:31.914666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18ab1673e1f84248a53ff27b3ccf0257', 'request_id': 'req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:31.926342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18ab1673e1f84248a53ff27b3ccf0257', 'request_id': 'req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:31.942982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] The token cache is being invalidate because project 18ab1673e1f84248a53ff27b3ccf0257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:31.943418 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:31.944497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:31.950844 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 824/1654] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:31 2022] DELETE /identity/v3/projects/18ab1673e1f84248a53ff27b3ccf0257 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:31.952815 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087eec46-c4c9-4eaa-8213-3f8adc6da52c service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:31.971262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087eec46-c4c9-4eaa-8213-3f8adc6da52c service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:32.148005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:32.148705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:32.148966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:32.149240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] PATH_INFO: `/v3/users/3bf1a6a52945427c84d2364d29350a21` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:32.161184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bf1a6a52945427c84d2364d29350a21)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:32.163673 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:32.230142 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:32.298495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bf1a6a52945427c84d2364d29350a21'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:32.309992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bf1a6a52945427c84d2364d29350a21'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:32.310644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bf1a6a52945427c84d2364d29350a21'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:32.311278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bf1a6a52945427c84d2364d29350a21'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:32.326287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bf1a6a52945427c84d2364d29350a21'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:32.370552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 831/1655] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:52:31 2022] DELETE /identity/v3/users/3bf1a6a52945427c84d2364d29350a21 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:32.973821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:32.976040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:33.008587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbdc27-78b9-41a0-9e1d-891edaaa9a21 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:33.222583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd750f63-568b-49bf-8f0c-4ef8191de90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:33.226185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd750f63-568b-49bf-8f0c-4ef8191de90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:33.226185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd750f63-568b-49bf-8f0c-4ef8191de90c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:33.229610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd750f63-568b-49bf-8f0c-4ef8191de90c admin admin] PATH_INFO: `/v3/users/e13c9bee1a1c4a6b8a215cfba256c51e` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:33.231815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd750f63-568b-49bf-8f0c-4ef8191de90c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e13c9bee1a1c4a6b8a215cfba256c51e)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:33.233380 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: WARNING py.warnings [None req-bd750f63-568b-49bf-8f0c-4ef8191de90c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:52:33.233380 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: warnings.warn(msg) Sep 08 13:52:33.233380 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: Sep 08 13:52:33.236511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd750f63-568b-49bf-8f0c-4ef8191de90c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:33.316481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:33.318031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:33.346850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829bf0c-ef64-490d-87bd-2e4aeece956a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:33.370491 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:33.372579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:33.372904 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:33.373185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] PATH_INFO: `/v3/users/cbea503b4cca4fb886179ff3e0564301` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:33.374835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbea503b4cca4fb886179ff3e0564301)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:33.391472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:33.405298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bd750f63-568b-49bf-8f0c-4ef8191de90c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e13c9bee1a1c4a6b8a215cfba256c51e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:33.418015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bd750f63-568b-49bf-8f0c-4ef8191de90c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e13c9bee1a1c4a6b8a215cfba256c51e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:33.419043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bd750f63-568b-49bf-8f0c-4ef8191de90c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e13c9bee1a1c4a6b8a215cfba256c51e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:33.419930 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bd750f63-568b-49bf-8f0c-4ef8191de90c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e13c9bee1a1c4a6b8a215cfba256c51e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:33.443959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bd750f63-568b-49bf-8f0c-4ef8191de90c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e13c9bee1a1c4a6b8a215cfba256c51e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:33.543291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 825/1656] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:52:32 2022] DELETE /identity/v3/users/e13c9bee1a1c4a6b8a215cfba256c51e => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:33.559513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbea503b4cca4fb886179ff3e0564301'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:33.587849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbea503b4cca4fb886179ff3e0564301'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:33.588988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbea503b4cca4fb886179ff3e0564301'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:33.589929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbea503b4cca4fb886179ff3e0564301'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:33.609432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbea503b4cca4fb886179ff3e0564301'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:33.663503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 832/1657] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:52:33 2022] DELETE /identity/v3/users/cbea503b4cca4fb886179ff3e0564301 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:52:33.667960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3128c-a584-4311-a45d-1443e7290871 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:33.668258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3128c-a584-4311-a45d-1443e7290871 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:33.668537 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3128c-a584-4311-a45d-1443e7290871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:33.707250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-78d3128c-a584-4311-a45d-1443e7290871 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:33.708492 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-78d3128c-a584-4311-a45d-1443e7290871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:33.708671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:33.710419 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:33.733489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:33.866310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 826/1658] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:33.870511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37114e21-b5d7-45da-8788-13de3970ff92 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:33.875499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:33.876226 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:33.876579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:33.876919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] PATH_INFO: `/v3/projects/b2b7fd2f4a4e478bb57d996da77bb2cd` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:33.888590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b7fd2f4a4e478bb57d996da77bb2cd)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:33.890038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: WARNING py.warnings [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:52:33.890038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: warnings.warn(msg) Sep 08 13:52:33.890038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: Sep 08 13:52:33.891267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:33.934128 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:33.996943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b7fd2f4a4e478bb57d996da77bb2cd', 'request_id': 'req-b437391e-d00f-4e0f-a86c-f030ebff7087'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:34.054346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b7fd2f4a4e478bb57d996da77bb2cd', 'request_id': 'req-b437391e-d00f-4e0f-a86c-f030ebff7087'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:34.063870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b7fd2f4a4e478bb57d996da77bb2cd', 'request_id': 'req-b437391e-d00f-4e0f-a86c-f030ebff7087'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:34.082882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] The token cache is being invalidate because project b2b7fd2f4a4e478bb57d996da77bb2cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:34.083190 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:34.087774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:34.090287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 833/1659] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:33 2022] DELETE /identity/v3/projects/b2b7fd2f4a4e478bb57d996da77bb2cd => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:34.096484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5616d14-6560-4830-a05e-41a9c19190aa None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:34.097273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5616d14-6560-4830-a05e-41a9c19190aa None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:34.097798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5616d14-6560-4830-a05e-41a9c19190aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:34.142775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b5616d14-6560-4830-a05e-41a9c19190aa None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:34.144385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b5616d14-6560-4830-a05e-41a9c19190aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:34.196682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:34.198701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:34.309513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 827/1660] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:34.311276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8968c6-5885-41d6-a6eb-7b645cfcbf67 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:34.312240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8968c6-5885-41d6-a6eb-7b645cfcbf67 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:34.312668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8968c6-5885-41d6-a6eb-7b645cfcbf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:34.354230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4c8968c6-5885-41d6-a6eb-7b645cfcbf67 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:34.355591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4c8968c6-5885-41d6-a6eb-7b645cfcbf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:34.362768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b437391e-d00f-4e0f-a86c-f030ebff7087 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:34.377145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c33120-4789-4986-a1b1-ae85899afd0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:34.377794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 828/1661] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:34.378314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c33120-4789-4986-a1b1-ae85899afd0a service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:34.378901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c33120-4789-4986-a1b1-ae85899afd0a service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:34.379374 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c33120-4789-4986-a1b1-ae85899afd0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:34.382085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8968c6-5885-41d6-a6eb-7b645cfcbf67 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:34.397887 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b4c33120-4789-4986-a1b1-ae85899afd0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:34.400363 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8968c6-5885-41d6-a6eb-7b645cfcbf67 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:34.413506 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abee29-97be-485c-b812-5b452d907752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:34.414509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abee29-97be-485c-b812-5b452d907752 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:34.414979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abee29-97be-485c-b812-5b452d907752 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:34.415431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61abee29-97be-485c-b812-5b452d907752 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:34.416129 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c33120-4789-4986-a1b1-ae85899afd0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:34.416460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abee29-97be-485c-b812-5b452d907752 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:34.416944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abee29-97be-485c-b812-5b452d907752 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:34.418750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61abee29-97be-485c-b812-5b452d907752 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:34.428337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c33120-4789-4986-a1b1-ae85899afd0a service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:34.451737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 829/1662] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:52:34.453649 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 834/1663] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:52:34.454149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abee29-97be-485c-b812-5b452d907752 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:34.476163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61abee29-97be-485c-b812-5b452d907752 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:34.488285 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85f4e5-5a1f-44f0-99c6-a87931115d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:34.488893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85f4e5-5a1f-44f0-99c6-a87931115d24 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:34.489177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85f4e5-5a1f-44f0-99c6-a87931115d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:34.489452 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85f4e5-5a1f-44f0-99c6-a87931115d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:34.490132 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85f4e5-5a1f-44f0-99c6-a87931115d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:34.492083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e85f4e5-5a1f-44f0-99c6-a87931115d24 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:34.552945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 830/1664] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:34 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:34.557045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c33120-4789-4986-a1b1-ae85899afd0a service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:34.572833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c33120-4789-4986-a1b1-ae85899afd0a service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:34.729694 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95bd5c-fdb6-4988-8049-5aad1869013e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:34.730857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95bd5c-fdb6-4988-8049-5aad1869013e admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:34.731330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95bd5c-fdb6-4988-8049-5aad1869013e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:34.731800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95bd5c-fdb6-4988-8049-5aad1869013e admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:34.732632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95bd5c-fdb6-4988-8049-5aad1869013e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:34.735365 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95bd5c-fdb6-4988-8049-5aad1869013e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:34.767775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 835/1665] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:34 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:34.771754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85f4e5-5a1f-44f0-99c6-a87931115d24 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:34.786751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85f4e5-5a1f-44f0-99c6-a87931115d24 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:34.798688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ef214-c78e-43ea-80a0-77e2271a7cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:34.799197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ef214-c78e-43ea-80a0-77e2271a7cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:34.799484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ef214-c78e-43ea-80a0-77e2271a7cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:34.799762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ef214-c78e-43ea-80a0-77e2271a7cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:34.800424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ef214-c78e-43ea-80a0-77e2271a7cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:34.802174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ef214-c78e-43ea-80a0-77e2271a7cf2 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:34.833408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95bd5c-fdb6-4988-8049-5aad1869013e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:34.835338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 831/1666] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:34.850239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95bd5c-fdb6-4988-8049-5aad1869013e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:34.862859 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c728746-618d-498f-8fb9-07939252850b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:34.863752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c728746-618d-498f-8fb9-07939252850b admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:34.864233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c728746-618d-498f-8fb9-07939252850b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:34.864934 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c728746-618d-498f-8fb9-07939252850b admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:34.866872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c728746-618d-498f-8fb9-07939252850b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:34.871165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c728746-618d-498f-8fb9-07939252850b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:34.888916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 836/1667] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:52:34.893016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ef214-c78e-43ea-80a0-77e2271a7cf2 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:34.905250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ef214-c78e-43ea-80a0-77e2271a7cf2 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:34.912272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f6cce-6f7a-4a40-ba92-a3dd43b8928c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:34.913104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f6cce-6f7a-4a40-ba92-a3dd43b8928c admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:34.913468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f6cce-6f7a-4a40-ba92-a3dd43b8928c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:34.913882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f6cce-6f7a-4a40-ba92-a3dd43b8928c admin admin] PATH_INFO: `/v3/projects/18e9b0ebe369463780030429234b202b/users/8fa80058381045d6af75036716c834d7/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:34.961662 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f6cce-6f7a-4a40-ba92-a3dd43b8928c admin admin] RBAC: Authorizing `identity:create_grant(project_id=18e9b0ebe369463780030429234b202b, user_id=8fa80058381045d6af75036716c834d7, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:34.971592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f6cce-6f7a-4a40-ba92-a3dd43b8928c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:35.005778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 832/1668] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:34 2022] PUT /identity/v3/projects/18e9b0ebe369463780030429234b202b/users/8fa80058381045d6af75036716c834d7/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:35.017650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7169a8-71a9-4f65-adfe-833da9b59b5c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:35.018090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7169a8-71a9-4f65-adfe-833da9b59b5c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:35.019269 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7169a8-71a9-4f65-adfe-833da9b59b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:35.089094 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1e7169a8-71a9-4f65-adfe-833da9b59b5c None None] MFA Rules not processed for user `8fa80058381045d6af75036716c834d7`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:35.092144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1e7169a8-71a9-4f65-adfe-833da9b59b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:35.130128 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:35.289689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f6cce-6f7a-4a40-ba92-a3dd43b8928c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:35.291787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e43f6cce-6f7a-4a40-ba92-a3dd43b8928c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:35.304011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 837/1669] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:35.305068 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7169a8-71a9-4f65-adfe-833da9b59b5c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:35.313766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1e7169a8-71a9-4f65-adfe-833da9b59b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:35.440355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:35.537013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7169a8-71a9-4f65-adfe-833da9b59b5c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:35.539799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f6cce-6f7a-4a40-ba92-a3dd43b8928c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:35.547450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:35.548278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:35.548566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:35.548892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f admin admin] PATH_INFO: `/v3/projects/1a35514109a1451a81faa1e8eeee4362` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:35.550302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a35514109a1451a81faa1e8eeee4362)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:35.552277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:35.554751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64750039-cd62-4038-8f2e-4054530affcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:35.555756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64750039-cd62-4038-8f2e-4054530affcc service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:35.556435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64750039-cd62-4038-8f2e-4054530affcc service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:35.558554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64750039-cd62-4038-8f2e-4054530affcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:35.595023 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64750039-cd62-4038-8f2e-4054530affcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:35.599639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64750039-cd62-4038-8f2e-4054530affcc service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:35.646860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 833/1670] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:52:35.657006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64750039-cd62-4038-8f2e-4054530affcc service nova] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:35.661679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-64750039-cd62-4038-8f2e-4054530affcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:35.669497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a35514109a1451a81faa1e8eeee4362', 'request_id': 'req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:35.685495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64750039-cd62-4038-8f2e-4054530affcc service nova] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:35.724581 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a35514109a1451a81faa1e8eeee4362', 'request_id': 'req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:35.772921 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a35514109a1451a81faa1e8eeee4362', 'request_id': 'req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:35.792214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:35.793409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:35.803286 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:35.803880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] PATH_INFO: `/v3/projects/98c2fd35f46c4314874bcd6c09c08d4f` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:35.807377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98c2fd35f46c4314874bcd6c09c08d4f)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:35.814773 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f admin admin] The token cache is being invalidate because project 1a35514109a1451a81faa1e8eeee4362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:35.815372 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:35.815687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:35.817287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-732c2e16-9b5c-4ab8-a466-7deb1c8b916f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:35.819796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 838/1671] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:35 2022] DELETE /identity/v3/projects/1a35514109a1451a81faa1e8eeee4362 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:35.828133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53062e33-6447-4057-a4b6-ee084be8de64 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:35.829825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53062e33-6447-4057-a4b6-ee084be8de64 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:35.830335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53062e33-6447-4057-a4b6-ee084be8de64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:35.920061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-53062e33-6447-4057-a4b6-ee084be8de64 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:35.924718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-53062e33-6447-4057-a4b6-ee084be8de64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:35.936501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 839/1672] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:35.949660 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c520a-b150-447f-b729-0493c09d1680 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:35.953185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c520a-b150-447f-b729-0493c09d1680 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:35.955065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c520a-b150-447f-b729-0493c09d1680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:35.991760 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98c2fd35f46c4314874bcd6c09c08d4f', 'request_id': 'req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.001228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98c2fd35f46c4314874bcd6c09c08d4f', 'request_id': 'req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.030092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98c2fd35f46c4314874bcd6c09c08d4f', 'request_id': 'req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.058208 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] The token cache is being invalidate because project 98c2fd35f46c4314874bcd6c09c08d4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:36.058363 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-587c520a-b150-447f-b729-0493c09d1680 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:36.058597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.059356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.059465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-587c520a-b150-447f-b729-0493c09d1680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:36.061207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 834/1673] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:35 2022] DELETE /identity/v3/projects/98c2fd35f46c4314874bcd6c09c08d4f => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:36.065468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:36.067042 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:36.148085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:36.233603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 840/1674] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:36.244726 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c520a-b150-447f-b729-0493c09d1680 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:36.246601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:36.249368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-587c520a-b150-447f-b729-0493c09d1680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:36.258757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8a1d6-cad5-4fef-b3f3-e65e40fddab8 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:36.276598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:36.277077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:36.277365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:36.281345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] PATH_INFO: `/v3/users/8fa80058381045d6af75036716c834d7` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:36.285912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fa80058381045d6af75036716c834d7)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:36.288575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:36.290994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c520a-b150-447f-b729-0493c09d1680 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:36.303380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:36.308644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:36.308979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:36.309264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] PATH_INFO: `/v3/users/26c6b72729a24bf4831b2964524af902` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:36.313603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c6b72729a24bf4831b2964524af902)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:36.316244 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:36.392391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fa80058381045d6af75036716c834d7'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.406838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fa80058381045d6af75036716c834d7'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.407554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fa80058381045d6af75036716c834d7'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.408185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fa80058381045d6af75036716c834d7'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.419778 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c6b72729a24bf4831b2964524af902'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.420145 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fa80058381045d6af75036716c834d7'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.430674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c6b72729a24bf4831b2964524af902'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.431573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c6b72729a24bf4831b2964524af902'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.432113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c6b72729a24bf4831b2964524af902'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.447305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c6b72729a24bf4831b2964524af902'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.448771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 835/1675] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:52:36 2022] DELETE /identity/v3/users/8fa80058381045d6af75036716c834d7 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:36.449754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:36.451946 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:36.489442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 841/1676] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:52:36 2022] DELETE /identity/v3/users/26c6b72729a24bf4831b2964524af902 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:36.490345 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:36.496517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:36.581368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8081d8b-2fb4-411f-a7ca-9c1afaada19b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:36.637755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50bcc2-1279-46fd-8ede-9533d1244897 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:36.651698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f0275-68cc-47ed-9dc5-9844cbd9efc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:36.652373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f0275-68cc-47ed-9dc5-9844cbd9efc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:36.652877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f0275-68cc-47ed-9dc5-9844cbd9efc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:36.652989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f0275-68cc-47ed-9dc5-9844cbd9efc6 admin admin] PATH_INFO: `/v3/projects/8a4eb5c1f5b44cfab4a3869cc0a06851` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:36.661768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f0275-68cc-47ed-9dc5-9844cbd9efc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a4eb5c1f5b44cfab4a3869cc0a06851)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:36.661768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f0275-68cc-47ed-9dc5-9844cbd9efc6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:36.678430 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91eced2-1579-491e-b954-e550f40458bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:36.679797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91eced2-1579-491e-b954-e550f40458bc admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:36.680084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91eced2-1579-491e-b954-e550f40458bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:36.680356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91eced2-1579-491e-b954-e550f40458bc admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:36.681130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91eced2-1579-491e-b954-e550f40458bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:36.681431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91eced2-1579-491e-b954-e550f40458bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:36.683740 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91eced2-1579-491e-b954-e550f40458bc admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:36.700522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 836/1677] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:36.702098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf169b9f-bb07-4045-bd60-21f57088ccf4 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:36.702930 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf169b9f-bb07-4045-bd60-21f57088ccf4 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:36.703433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf169b9f-bb07-4045-bd60-21f57088ccf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:36.755153 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-288f0275-68cc-47ed-9dc5-9844cbd9efc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a4eb5c1f5b44cfab4a3869cc0a06851', 'request_id': 'req-288f0275-68cc-47ed-9dc5-9844cbd9efc6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.757363 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:36.773062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-288f0275-68cc-47ed-9dc5-9844cbd9efc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a4eb5c1f5b44cfab4a3869cc0a06851', 'request_id': 'req-288f0275-68cc-47ed-9dc5-9844cbd9efc6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.784597 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-288f0275-68cc-47ed-9dc5-9844cbd9efc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a4eb5c1f5b44cfab4a3869cc0a06851', 'request_id': 'req-288f0275-68cc-47ed-9dc5-9844cbd9efc6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.796784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-cf169b9f-bb07-4045-bd60-21f57088ccf4 None None] MFA Rules not processed for user `55d8e06905644ac39777328db72526ca`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:36.800763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-cf169b9f-bb07-4045-bd60-21f57088ccf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:36.803395 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-288f0275-68cc-47ed-9dc5-9844cbd9efc6 admin admin] The token cache is being invalidate because project 8a4eb5c1f5b44cfab4a3869cc0a06851 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:36.803698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-288f0275-68cc-47ed-9dc5-9844cbd9efc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.805543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-288f0275-68cc-47ed-9dc5-9844cbd9efc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:36.809316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 842/1678] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:36 2022] DELETE /identity/v3/projects/8a4eb5c1f5b44cfab4a3869cc0a06851 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:36.810631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307867dc-3d78-48f6-a8a6-d36248b1e0fd None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:36.810903 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307867dc-3d78-48f6-a8a6-d36248b1e0fd None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:36.811179 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307867dc-3d78-48f6-a8a6-d36248b1e0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:36.818638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:36.861080 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-307867dc-3d78-48f6-a8a6-d36248b1e0fd None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:36.862379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-307867dc-3d78-48f6-a8a6-d36248b1e0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:36.919886 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 837/1679] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:36.920980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf169b9f-bb07-4045-bd60-21f57088ccf4 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:36.923058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-cf169b9f-bb07-4045-bd60-21f57088ccf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:37.019659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 843/1680] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:37.020996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc22c9-3924-4afe-a517-163738bb49c1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:37.021935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc22c9-3924-4afe-a517-163738bb49c1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:37.022640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc22c9-3924-4afe-a517-163738bb49c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:37.066537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-42cc22c9-3924-4afe-a517-163738bb49c1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:37.068034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-42cc22c9-3924-4afe-a517-163738bb49c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:37.071874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf169b9f-bb07-4045-bd60-21f57088ccf4 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:37.081630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30452a4e-65f3-47d0-820c-f505dc7f9235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:37.089682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30452a4e-65f3-47d0-820c-f505dc7f9235 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:37.089682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30452a4e-65f3-47d0-820c-f505dc7f9235 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:37.089682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30452a4e-65f3-47d0-820c-f505dc7f9235 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:37.091359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 844/1681] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:37.093351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9d672-3af7-43a5-a4c8-1314ef647e23 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:37.093868 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9d672-3af7-43a5-a4c8-1314ef647e23 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:37.094052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30452a4e-65f3-47d0-820c-f505dc7f9235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:37.094495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9d672-3af7-43a5-a4c8-1314ef647e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:37.096024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30452a4e-65f3-47d0-820c-f505dc7f9235 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:37.136737 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2bc9d672-3af7-43a5-a4c8-1314ef647e23 None None] MFA Rules not processed for user `55d8e06905644ac39777328db72526ca`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:37.140157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2bc9d672-3af7-43a5-a4c8-1314ef647e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:37.160282 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 838/1682] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:36 2022] POST /identity/v3/projects => generated 379 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:37.161516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca2402-3b12-4405-90f4-a815490a58b1 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:37.161845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca2402-3b12-4405-90f4-a815490a58b1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:37.162134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca2402-3b12-4405-90f4-a815490a58b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:37.167547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 845/1683] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:37.169465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9d672-3af7-43a5-a4c8-1314ef647e23 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:37.172752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2bc9d672-3af7-43a5-a4c8-1314ef647e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:37.210590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-0cca2402-3b12-4405-90f4-a815490a58b1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:37.211864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0cca2402-3b12-4405-90f4-a815490a58b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:37.353043 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9d672-3af7-43a5-a4c8-1314ef647e23 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:37.429255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 839/1684] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:37.431957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca2402-3b12-4405-90f4-a815490a58b1 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:37.462769 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca2402-3b12-4405-90f4-a815490a58b1 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:37.480641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b5948-4811-450e-aa55-3293b0922a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:37.482820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b5948-4811-450e-aa55-3293b0922a23 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:37.483229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b5948-4811-450e-aa55-3293b0922a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:37.483532 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b5948-4811-450e-aa55-3293b0922a23 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:37.486120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b5948-4811-450e-aa55-3293b0922a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:37.491550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b5948-4811-450e-aa55-3293b0922a23 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:37.496035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1390c3e-cf23-4ebd-846d-81f89b0f6237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:37.497102 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1390c3e-cf23-4ebd-846d-81f89b0f6237 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:37.497592 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1390c3e-cf23-4ebd-846d-81f89b0f6237 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:37.498391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1390c3e-cf23-4ebd-846d-81f89b0f6237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:37.508930 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1390c3e-cf23-4ebd-846d-81f89b0f6237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:37.510767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1390c3e-cf23-4ebd-846d-81f89b0f6237 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:37.526614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 846/1685] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:37.528034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1390c3e-cf23-4ebd-846d-81f89b0f6237 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:37.531692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c1390c3e-cf23-4ebd-846d-81f89b0f6237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:37.555046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 840/1686] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:37 2022] POST /identity/v3/users => generated 290 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:37.556928 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b5948-4811-450e-aa55-3293b0922a23 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:37.590320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b5948-4811-450e-aa55-3293b0922a23 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:37.595327 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:37.613981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fa7b9-bd2c-45cc-86de-15bc6545142b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:37.614615 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fa7b9-bd2c-45cc-86de-15bc6545142b admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:37.614893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fa7b9-bd2c-45cc-86de-15bc6545142b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:37.615161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fa7b9-bd2c-45cc-86de-15bc6545142b admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:37.615825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fa7b9-bd2c-45cc-86de-15bc6545142b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:37.619507 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fa7b9-bd2c-45cc-86de-15bc6545142b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:37.628186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fa7b9-bd2c-45cc-86de-15bc6545142b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:37.659216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 841/1687] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:37.666960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fa7b9-bd2c-45cc-86de-15bc6545142b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:37.682832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:37.692765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fa7b9-bd2c-45cc-86de-15bc6545142b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:37.708752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6cffc-842c-4cba-9528-1aa8f05fbc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:37.708752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6cffc-842c-4cba-9528-1aa8f05fbc6b admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:37.708752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6cffc-842c-4cba-9528-1aa8f05fbc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:37.708752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb6cffc-842c-4cba-9528-1aa8f05fbc6b admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:37.714166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1390c3e-cf23-4ebd-846d-81f89b0f6237 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:37.714862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6cffc-842c-4cba-9528-1aa8f05fbc6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:37.714862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb6cffc-842c-4cba-9528-1aa8f05fbc6b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:37.721428 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3845e4-2917-405e-b12f-962976b11ba0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:37.722254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3845e4-2917-405e-b12f-962976b11ba0 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:37.725011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3845e4-2917-405e-b12f-962976b11ba0 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:37.725434 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3845e4-2917-405e-b12f-962976b11ba0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:37.749104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 842/1688] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:37.750599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6cffc-842c-4cba-9528-1aa8f05fbc6b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:37.777522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb6cffc-842c-4cba-9528-1aa8f05fbc6b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:37.792467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3421f-2fcc-4f23-b901-274b38df78e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:37.793122 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3421f-2fcc-4f23-b901-274b38df78e1 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:37.794723 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3421f-2fcc-4f23-b901-274b38df78e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:37.795005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3421f-2fcc-4f23-b901-274b38df78e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:37.796950 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3421f-2fcc-4f23-b901-274b38df78e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:37.801774 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3421f-2fcc-4f23-b901-274b38df78e1 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:37.874025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 843/1689] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:37 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:37.875429 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3421f-2fcc-4f23-b901-274b38df78e1 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:37.890514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3845e4-2917-405e-b12f-962976b11ba0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:37.892302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3845e4-2917-405e-b12f-962976b11ba0 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:37.913495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 847/1690] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:37.914576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3845e4-2917-405e-b12f-962976b11ba0 service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:37.961879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3845e4-2917-405e-b12f-962976b11ba0 service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:38.121445 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3421f-2fcc-4f23-b901-274b38df78e1 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:38.143203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1963b-4b33-49cd-a8a7-e0a82a3c36f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:38.144105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1963b-4b33-49cd-a8a7-e0a82a3c36f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:38.144105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1963b-4b33-49cd-a8a7-e0a82a3c36f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:38.144387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1963b-4b33-49cd-a8a7-e0a82a3c36f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:38.145060 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b1963b-4b33-49cd-a8a7-e0a82a3c36f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:38.146824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b1963b-4b33-49cd-a8a7-e0a82a3c36f7 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:38.153412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75648147-3c7e-4b9e-beeb-498702a2fde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:38.154209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75648147-3c7e-4b9e-beeb-498702a2fde8 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:38.154621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75648147-3c7e-4b9e-beeb-498702a2fde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:38.159683 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75648147-3c7e-4b9e-beeb-498702a2fde8 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:38.162628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75648147-3c7e-4b9e-beeb-498702a2fde8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:38.167419 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75648147-3c7e-4b9e-beeb-498702a2fde8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:38.192130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 844/1691] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:38.193168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1963b-4b33-49cd-a8a7-e0a82a3c36f7 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:38.207806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-33b1963b-4b33-49cd-a8a7-e0a82a3c36f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:38.258435 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 848/1692] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:37 2022] POST /identity/v3/users => generated 286 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:38.259352 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75648147-3c7e-4b9e-beeb-498702a2fde8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:38.260146 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1963b-4b33-49cd-a8a7-e0a82a3c36f7 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:38.277349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354695a9-5caa-4d70-a8cc-f216675d9a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:38.278054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354695a9-5caa-4d70-a8cc-f216675d9a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:38.278348 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354695a9-5caa-4d70-a8cc-f216675d9a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:38.278603 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354695a9-5caa-4d70-a8cc-f216675d9a8e admin admin] PATH_INFO: `/v3/projects/8d113649d43f48c499dd573c9880c5de` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:38.282964 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354695a9-5caa-4d70-a8cc-f216675d9a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d113649d43f48c499dd573c9880c5de)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:38.285894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354695a9-5caa-4d70-a8cc-f216675d9a8e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:38.291876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75648147-3c7e-4b9e-beeb-498702a2fde8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:38.318830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093a930-3f8d-4727-ad3a-552b70ba197c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:38.319648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093a930-3f8d-4727-ad3a-552b70ba197c admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:38.322467 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093a930-3f8d-4727-ad3a-552b70ba197c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:38.324925 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093a930-3f8d-4727-ad3a-552b70ba197c admin admin] PATH_INFO: `/v3/projects/ef2a5c29fabc4219809253a7cfb7b26a/users/bb44795a0a8b4f8189e82b21826c96b2/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:38.345581 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093a930-3f8d-4727-ad3a-552b70ba197c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2a5c29fabc4219809253a7cfb7b26a, user_id=bb44795a0a8b4f8189e82b21826c96b2, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:38.348683 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093a930-3f8d-4727-ad3a-552b70ba197c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:38.375533 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 849/1693] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:38 2022] PUT /identity/v3/projects/ef2a5c29fabc4219809253a7cfb7b26a/users/bb44795a0a8b4f8189e82b21826c96b2/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:38.376830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093a930-3f8d-4727-ad3a-552b70ba197c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:38.415993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-354695a9-5caa-4d70-a8cc-f216675d9a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d113649d43f48c499dd573c9880c5de', 'request_id': 'req-354695a9-5caa-4d70-a8cc-f216675d9a8e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:38.430858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-354695a9-5caa-4d70-a8cc-f216675d9a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d113649d43f48c499dd573c9880c5de', 'request_id': 'req-354695a9-5caa-4d70-a8cc-f216675d9a8e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:38.443394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-354695a9-5caa-4d70-a8cc-f216675d9a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d113649d43f48c499dd573c9880c5de', 'request_id': 'req-354695a9-5caa-4d70-a8cc-f216675d9a8e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:38.450936 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:38.465988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-354695a9-5caa-4d70-a8cc-f216675d9a8e admin admin] The token cache is being invalidate because project 8d113649d43f48c499dd573c9880c5de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:38.466444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-354695a9-5caa-4d70-a8cc-f216675d9a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:38.470210 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-354695a9-5caa-4d70-a8cc-f216675d9a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:38.477932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 845/1694] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:38 2022] DELETE /identity/v3/projects/8d113649d43f48c499dd573c9880c5de => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:38.479896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c236321-7dad-4e07-9ee6-bd7e569dbec7 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:38.480185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c236321-7dad-4e07-9ee6-bd7e569dbec7 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:38.480466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c236321-7dad-4e07-9ee6-bd7e569dbec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:38.532647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093a930-3f8d-4727-ad3a-552b70ba197c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:38.559372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1c236321-7dad-4e07-9ee6-bd7e569dbec7 None None] MFA Rules not processed for user `bb44795a0a8b4f8189e82b21826c96b2`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:38.560614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1c236321-7dad-4e07-9ee6-bd7e569dbec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:38.645438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:38.689494 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b06487-3881-4606-9235-e564aba04a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:38.690532 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b06487-3881-4606-9235-e564aba04a25 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:38.691019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b06487-3881-4606-9235-e564aba04a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:38.691742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b06487-3881-4606-9235-e564aba04a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:38.692123 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b06487-3881-4606-9235-e564aba04a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:38.701775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b06487-3881-4606-9235-e564aba04a25 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:38.726296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 846/1695] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:38.727643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1098dfc-f764-412c-a78a-cf0b8135dd55 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:38.727932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1098dfc-f764-412c-a78a-cf0b8135dd55 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:38.728213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1098dfc-f764-412c-a78a-cf0b8135dd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:38.749991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 850/1696] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:38.750949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b06487-3881-4606-9235-e564aba04a25 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:38.761849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b4b06487-3881-4606-9235-e564aba04a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:38.798519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b06487-3881-4606-9235-e564aba04a25 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:38.814768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e3d19-39c7-4af2-b5c7-d874bca154ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:38.818254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e3d19-39c7-4af2-b5c7-d874bca154ab admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:38.818552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e3d19-39c7-4af2-b5c7-d874bca154ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:38.819028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e3d19-39c7-4af2-b5c7-d874bca154ab admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:38.821314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e3d19-39c7-4af2-b5c7-d874bca154ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:38.827013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e3d19-39c7-4af2-b5c7-d874bca154ab admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:38.836776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b1098dfc-f764-412c-a78a-cf0b8135dd55 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:38.839267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b1098dfc-f764-412c-a78a-cf0b8135dd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:38.864912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 847/1697] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:38.865695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1098dfc-f764-412c-a78a-cf0b8135dd55 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:38.873490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b1098dfc-f764-412c-a78a-cf0b8135dd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:38.900955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1098dfc-f764-412c-a78a-cf0b8135dd55 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:38.918570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 851/1698] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:38 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:38.925192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcfec6-e992-429f-98e7-cc9c62ae89f3 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:38.925192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcfec6-e992-429f-98e7-cc9c62ae89f3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:38.925449 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcfec6-e992-429f-98e7-cc9c62ae89f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:38.925638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2feb6f-560f-4d33-bfd2-703b4c9d6148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:38.926357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2feb6f-560f-4d33-bfd2-703b4c9d6148 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:38.926794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2feb6f-560f-4d33-bfd2-703b4c9d6148 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:38.927088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2feb6f-560f-4d33-bfd2-703b4c9d6148 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:38.927757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2feb6f-560f-4d33-bfd2-703b4c9d6148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:38.929374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2feb6f-560f-4d33-bfd2-703b4c9d6148 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:38.953550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 848/1699] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:38.957226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2feb6f-560f-4d33-bfd2-703b4c9d6148 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:38.986824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2feb6f-560f-4d33-bfd2-703b4c9d6148 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:38.990645 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b2dcfec6-e992-429f-98e7-cc9c62ae89f3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:38.995491 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b2dcfec6-e992-429f-98e7-cc9c62ae89f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:39.122736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 852/1700] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:39.123387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d648fc0-42c1-40cd-9313-942c45d8655c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:39.123523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcfec6-e992-429f-98e7-cc9c62ae89f3 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:39.124807 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d648fc0-42c1-40cd-9313-942c45d8655c admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:39.125231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d648fc0-42c1-40cd-9313-942c45d8655c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:39.125650 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d648fc0-42c1-40cd-9313-942c45d8655c admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:39.127342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d648fc0-42c1-40cd-9313-942c45d8655c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:39.132214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d648fc0-42c1-40cd-9313-942c45d8655c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:39.153464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcfec6-e992-429f-98e7-cc9c62ae89f3 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:39.169131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ccd91-e4bf-4948-a358-3cd77fb7ec38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:39.169952 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ccd91-e4bf-4948-a358-3cd77fb7ec38 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:39.170127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 849/1701] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:38 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:39.170237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ccd91-e4bf-4948-a358-3cd77fb7ec38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:39.170496 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ccd91-e4bf-4948-a358-3cd77fb7ec38 admin admin] PATH_INFO: `/v3/projects/c5084e7f21bb43479da70ba3ae82f89f/users/b4b7305e55ce4d62a4a2fcb8882c0f39/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:39.171553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d648fc0-42c1-40cd-9313-942c45d8655c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:39.184040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d648fc0-42c1-40cd-9313-942c45d8655c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:39.190630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72addfc6-7de2-413c-bda3-0285efd23438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:39.191471 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72addfc6-7de2-413c-bda3-0285efd23438 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:39.191920 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72addfc6-7de2-413c-bda3-0285efd23438 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:39.192622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72addfc6-7de2-413c-bda3-0285efd23438 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:39.194171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72addfc6-7de2-413c-bda3-0285efd23438 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:39.194660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72addfc6-7de2-413c-bda3-0285efd23438 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:39.196588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72addfc6-7de2-413c-bda3-0285efd23438 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:39.206126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ccd91-e4bf-4948-a358-3cd77fb7ec38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5084e7f21bb43479da70ba3ae82f89f, user_id=b4b7305e55ce4d62a4a2fcb8882c0f39, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:39.209563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ccd91-e4bf-4948-a358-3cd77fb7ec38 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:39.219429 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 850/1702] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:39.221238 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72addfc6-7de2-413c-bda3-0285efd23438 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:39.240802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 853/1703] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:39 2022] PUT /identity/v3/projects/c5084e7f21bb43479da70ba3ae82f89f/users/b4b7305e55ce4d62a4a2fcb8882c0f39/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:39.243296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ccd91-e4bf-4948-a358-3cd77fb7ec38 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:39.267923 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72addfc6-7de2-413c-bda3-0285efd23438 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:39.365870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:39.455805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ebfaa3-888b-41b1-badc-78dbcb722599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:39.456371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebfaa3-888b-41b1-badc-78dbcb722599 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:39.456636 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebfaa3-888b-41b1-badc-78dbcb722599 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:39.456917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ebfaa3-888b-41b1-badc-78dbcb722599 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:39.457590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ebfaa3-888b-41b1-badc-78dbcb722599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:39.461813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ebfaa3-888b-41b1-badc-78dbcb722599 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:39.467756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ccd91-e4bf-4948-a358-3cd77fb7ec38 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:39.481466 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13760e-14b6-45e4-97fa-707799310440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:39.484867 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13760e-14b6-45e4-97fa-707799310440 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:39.485268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13760e-14b6-45e4-97fa-707799310440 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:39.485663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13760e-14b6-45e4-97fa-707799310440 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:39.486468 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13760e-14b6-45e4-97fa-707799310440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:39.488796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13760e-14b6-45e4-97fa-707799310440 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:39.507687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 851/1704] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:39.512880 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ae19e-65f1-4679-a2eb-a7352f4f0771 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:39.512991 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ae19e-65f1-4679-a2eb-a7352f4f0771 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:39.513228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ae19e-65f1-4679-a2eb-a7352f4f0771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:39.566890 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 854/1705] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:39 2022] POST /identity/v3/projects => generated 357 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:39.571092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13760e-14b6-45e4-97fa-707799310440 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:39.580026 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-2d9ae19e-65f1-4679-a2eb-a7352f4f0771 None None] MFA Rules not processed for user `b4b7305e55ce4d62a4a2fcb8882c0f39`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:39.584777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2d9ae19e-65f1-4679-a2eb-a7352f4f0771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:39.689256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:39.787712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13760e-14b6-45e4-97fa-707799310440 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:39.793555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 852/1706] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:39.794756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ae19e-65f1-4679-a2eb-a7352f4f0771 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:39.799985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96deb0e0-50af-47fa-93c9-fd40abc06d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:39.804521 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96deb0e0-50af-47fa-93c9-fd40abc06d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:39.804823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96deb0e0-50af-47fa-93c9-fd40abc06d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:39.807817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96deb0e0-50af-47fa-93c9-fd40abc06d05 admin admin] PATH_INFO: `/v3/projects/2a7acd2e8ea543cba3094bf34c4d5af5/users/aa1596ff6299418db7ca7166795d435f/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:39.822863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ae19e-65f1-4679-a2eb-a7352f4f0771 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:39.839289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed257e-6498-4eb8-82de-0348a9bd512b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:39.839456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed257e-6498-4eb8-82de-0348a9bd512b admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:39.839878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed257e-6498-4eb8-82de-0348a9bd512b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:39.840219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed257e-6498-4eb8-82de-0348a9bd512b admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:39.840933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed257e-6498-4eb8-82de-0348a9bd512b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:39.845393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed257e-6498-4eb8-82de-0348a9bd512b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:39.847911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96deb0e0-50af-47fa-93c9-fd40abc06d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7acd2e8ea543cba3094bf34c4d5af5, user_id=aa1596ff6299418db7ca7166795d435f, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:39.855620 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96deb0e0-50af-47fa-93c9-fd40abc06d05 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:39.899102 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 855/1707] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:39 2022] PUT /identity/v3/projects/2a7acd2e8ea543cba3094bf34c4d5af5/users/aa1596ff6299418db7ca7166795d435f/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:39.899102 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96deb0e0-50af-47fa-93c9-fd40abc06d05 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:39.904023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-96deb0e0-50af-47fa-93c9-fd40abc06d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:39.912719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 853/1708] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:39 2022] POST /identity/v3/users => generated 279 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:39.917060 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed257e-6498-4eb8-82de-0348a9bd512b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:39.939644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed257e-6498-4eb8-82de-0348a9bd512b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:40.075998 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96deb0e0-50af-47fa-93c9-fd40abc06d05 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:40.084718 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d52ec-dcd2-477e-9438-88dce5ad2af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:40.086263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d52ec-dcd2-477e-9438-88dce5ad2af7 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.086936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d52ec-dcd2-477e-9438-88dce5ad2af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.087939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d52ec-dcd2-477e-9438-88dce5ad2af7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.088672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d52ec-dcd2-477e-9438-88dce5ad2af7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:40.089442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:40.090720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.092135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.093114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d52ec-dcd2-477e-9438-88dce5ad2af7 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:40.094303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] PATH_INFO: `/v3/projects/18e9b0ebe369463780030429234b202b` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.097659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=18e9b0ebe369463780030429234b202b)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:40.099593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:40.151841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 854/1709] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:39 2022] POST /identity/v3/projects => generated 373 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:40.154581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d52ec-dcd2-477e-9438-88dce5ad2af7 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:40.185241 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:40.204997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18e9b0ebe369463780030429234b202b', 'request_id': 'req-c658075d-9d16-489f-90a4-f3a453fb3c6f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:40.221923 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18e9b0ebe369463780030429234b202b', 'request_id': 'req-c658075d-9d16-489f-90a4-f3a453fb3c6f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:40.238870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18e9b0ebe369463780030429234b202b', 'request_id': 'req-c658075d-9d16-489f-90a4-f3a453fb3c6f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:40.259802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] The token cache is being invalidate because project 18e9b0ebe369463780030429234b202b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:40.260105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:40.260929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:40.263069 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 856/1710] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:39 2022] DELETE /identity/v3/projects/18e9b0ebe369463780030429234b202b => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:40.268144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:40.269727 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:40.364272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d52ec-dcd2-477e-9438-88dce5ad2af7 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:40.400848 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658075d-9d16-489f-90a4-f3a453fb3c6f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:40.408489 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a514df3-3a05-4628-abde-74922d3fcfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:40.410380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a514df3-3a05-4628-abde-74922d3fcfea admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.410380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a514df3-3a05-4628-abde-74922d3fcfea admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.410609 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a514df3-3a05-4628-abde-74922d3fcfea admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.413460 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a514df3-3a05-4628-abde-74922d3fcfea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:40.413864 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a514df3-3a05-4628-abde-74922d3fcfea admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:40.427437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:40.430249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 857/1711] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:40.431778 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cab41-2665-469f-8b5b-889956364efc None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.432299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cab41-2665-469f-8b5b-889956364efc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.432789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cab41-2665-469f-8b5b-889956364efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.470179 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dd617-644b-4a69-b86a-a50ed9871123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:40.471014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dd617-644b-4a69-b86a-a50ed9871123 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.471352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dd617-644b-4a69-b86a-a50ed9871123 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.471632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dd617-644b-4a69-b86a-a50ed9871123 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.472529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657dd617-644b-4a69-b86a-a50ed9871123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:40.474181 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657dd617-644b-4a69-b86a-a50ed9871123 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:40.475945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-249cab41-2665-469f-8b5b-889956364efc None None] MFA Rules not processed for user `2e48a8c1b91142cf9d1af17f28e41961`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:40.479746 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-249cab41-2665-469f-8b5b-889956364efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:40.494018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 855/1712] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:40.494018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dd617-644b-4a69-b86a-a50ed9871123 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:40.496685 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-657dd617-644b-4a69-b86a-a50ed9871123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:40.513297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dd617-644b-4a69-b86a-a50ed9871123 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:40.534597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed728ee-6836-4b9c-8e27-79fc2818d2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:40.535509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed728ee-6836-4b9c-8e27-79fc2818d2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.536092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed728ee-6836-4b9c-8e27-79fc2818d2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.536553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed728ee-6836-4b9c-8e27-79fc2818d2e6 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.537426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed728ee-6836-4b9c-8e27-79fc2818d2e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:40.540989 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed728ee-6836-4b9c-8e27-79fc2818d2e6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:40.590975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 856/1713] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:40 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:40.592557 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d487bc-5601-405c-95d7-05d5da6fcdcc None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.593007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d487bc-5601-405c-95d7-05d5da6fcdcc None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.593449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d487bc-5601-405c-95d7-05d5da6fcdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.621827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b3d487bc-5601-405c-95d7-05d5da6fcdcc None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:40.627866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b3d487bc-5601-405c-95d7-05d5da6fcdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:40.650262 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 857/1714] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:40.651505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d487bc-5601-405c-95d7-05d5da6fcdcc None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:40.669157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 858/1715] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:40.671650 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d487bc-5601-405c-95d7-05d5da6fcdcc None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:40.679747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cab41-2665-469f-8b5b-889956364efc None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:40.681176 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-249cab41-2665-469f-8b5b-889956364efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:40.684077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b93ceb6-832d-44d0-8ab2-53126bb3e3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:40.685009 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93ceb6-832d-44d0-8ab2-53126bb3e3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.685392 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93ceb6-832d-44d0-8ab2-53126bb3e3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.685805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93ceb6-832d-44d0-8ab2-53126bb3e3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.687250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93ceb6-832d-44d0-8ab2-53126bb3e3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:40.690114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b93ceb6-832d-44d0-8ab2-53126bb3e3f8 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:40.707829 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cab41-2665-469f-8b5b-889956364efc None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:40.708446 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 858/1716] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:40.710038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491a084-6b70-48ed-9f80-d27414705873 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.710728 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491a084-6b70-48ed-9f80-d27414705873 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.711289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491a084-6b70-48ed-9f80-d27414705873 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.713565 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 859/1717] 10.222.0.55 () {54 vars in 912 bytes} [Thu Sep 8 13:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:40.714482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491a084-6b70-48ed-9f80-d27414705873 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:40.723290 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68006278-27b5-4809-93dc-3b09c1305b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:40.723911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68006278-27b5-4809-93dc-3b09c1305b5c admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.724190 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68006278-27b5-4809-93dc-3b09c1305b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.724505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68006278-27b5-4809-93dc-3b09c1305b5c admin admin] PATH_INFO: `/v3/projects/2a7acd2e8ea543cba3094bf34c4d5af5/users/aa1596ff6299418db7ca7166795d435f/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.730263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491a084-6b70-48ed-9f80-d27414705873 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:40.730994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68006278-27b5-4809-93dc-3b09c1305b5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7acd2e8ea543cba3094bf34c4d5af5, user_id=aa1596ff6299418db7ca7166795d435f, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:40.736834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53352839-0a0e-48a5-9d6a-e5a8f7e6aa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:40.737602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68006278-27b5-4809-93dc-3b09c1305b5c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:40.738607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53352839-0a0e-48a5-9d6a-e5a8f7e6aa3f admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.739141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53352839-0a0e-48a5-9d6a-e5a8f7e6aa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.739966 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53352839-0a0e-48a5-9d6a-e5a8f7e6aa3f admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.740912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53352839-0a0e-48a5-9d6a-e5a8f7e6aa3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:40.743045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53352839-0a0e-48a5-9d6a-e5a8f7e6aa3f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:40.753531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 859/1718] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:40 2022] PUT /identity/v3/projects/2a7acd2e8ea543cba3094bf34c4d5af5/users/aa1596ff6299418db7ca7166795d435f/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:40.755655 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf9a72-28fb-4634-bf20-e9b28e4e6006 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.756200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf9a72-28fb-4634-bf20-e9b28e4e6006 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.756614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf9a72-28fb-4634-bf20-e9b28e4e6006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.757422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 860/1719] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:40.763044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53352839-0a0e-48a5-9d6a-e5a8f7e6aa3f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:40.767199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-53352839-0a0e-48a5-9d6a-e5a8f7e6aa3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:40.797384 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-18bf9a72-28fb-4634-bf20-e9b28e4e6006 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:40.800301 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-18bf9a72-28fb-4634-bf20-e9b28e4e6006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:40.882093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:40.932121 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53352839-0a0e-48a5-9d6a-e5a8f7e6aa3f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:40.957963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 860/1720] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:40.957963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf9a72-28fb-4634-bf20-e9b28e4e6006 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:40.972431 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf9a72-28fb-4634-bf20-e9b28e4e6006 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:40.978602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69358d-f037-4742-a613-e7fd1a939a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:40.979004 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69358d-f037-4742-a613-e7fd1a939a4b admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.979287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69358d-f037-4742-a613-e7fd1a939a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.980377 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69358d-f037-4742-a613-e7fd1a939a4b admin admin] PATH_INFO: `/v3/projects/4a0dc70d5a194b3a958ef5dfcab44533/users/9ab658a6718943fea13aae3eb0330247/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.009775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69358d-f037-4742-a613-e7fd1a939a4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a0dc70d5a194b3a958ef5dfcab44533, user_id=9ab658a6718943fea13aae3eb0330247, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:41.013140 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69358d-f037-4742-a613-e7fd1a939a4b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:41.038038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 861/1721] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:40 2022] PUT /identity/v3/projects/4a0dc70d5a194b3a958ef5dfcab44533/users/9ab658a6718943fea13aae3eb0330247/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:41.039348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f5219-854a-4a7b-a355-1962ed7a097c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.039638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f5219-854a-4a7b-a355-1962ed7a097c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.039913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f5219-854a-4a7b-a355-1962ed7a097c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.065034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31572e-6669-4390-873c-c570a7d3e1b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:41.066137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31572e-6669-4390-873c-c570a7d3e1b5 service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.066747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31572e-6669-4390-873c-c570a7d3e1b5 service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.067311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31572e-6669-4390-873c-c570a7d3e1b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.099373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:41.144563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d01f5219-854a-4a7b-a355-1962ed7a097c None None] MFA Rules not processed for user `531ebc79bf424e9b8762b2c7ec7fe8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:41.149190 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d01f5219-854a-4a7b-a355-1962ed7a097c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:41.182076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31572e-6669-4390-873c-c570a7d3e1b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:41.183718 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31572e-6669-4390-873c-c570a7d3e1b5 service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:41.198567 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 861/1722] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:41.211987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31572e-6669-4390-873c-c570a7d3e1b5 service nova] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:41.222832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 862/1723] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 13:52:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:41.229631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcad56-e078-4954-9e9c-e60734678bca None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.229961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcad56-e078-4954-9e9c-e60734678bca None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.230190 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dcad56-e078-4954-9e9c-e60734678bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.238007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31572e-6669-4390-873c-c570a7d3e1b5 service nova] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:41.324256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-05dcad56-e078-4954-9e9c-e60734678bca None None] MFA Rules not processed for user `aa1596ff6299418db7ca7166795d435f`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:41.326083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-05dcad56-e078-4954-9e9c-e60734678bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:41.397650 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabd928-e4da-4185-9862-bfade45e9204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:41.398570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabd928-e4da-4185-9862-bfade45e9204 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.398855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabd928-e4da-4185-9862-bfade45e9204 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.399129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabd928-e4da-4185-9862-bfade45e9204 admin admin] PATH_INFO: `/v3/projects/72927f397b1c4a259dc1414e8f3aacd3/users/20a38313912c4b57a288cf11c57f498c/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.434033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabd928-e4da-4185-9862-bfade45e9204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72927f397b1c4a259dc1414e8f3aacd3, user_id=20a38313912c4b57a288cf11c57f498c, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:41.437039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabd928-e4da-4185-9862-bfade45e9204 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:41.460644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 862/1724] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:41 2022] PUT /identity/v3/projects/72927f397b1c4a259dc1414e8f3aacd3/users/20a38313912c4b57a288cf11c57f498c/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:41.465598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabd928-e4da-4185-9862-bfade45e9204 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:41.467235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 863/1725] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:41.471998 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbce6b-f50e-4fe0-9ef5-a3605813f1d0 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.477521 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbce6b-f50e-4fe0-9ef5-a3605813f1d0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.477815 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbce6b-f50e-4fe0-9ef5-a3605813f1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.557160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-92cbce6b-f50e-4fe0-9ef5-a3605813f1d0 None None] MFA Rules not processed for user `9ab658a6718943fea13aae3eb0330247`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:41.563479 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-92cbce6b-f50e-4fe0-9ef5-a3605813f1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:41.586349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabd928-e4da-4185-9862-bfade45e9204 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:41.591985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a98109-cecd-44e8-8ecd-d0cd8297b43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:41.592801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a98109-cecd-44e8-8ecd-d0cd8297b43e admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.593840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a98109-cecd-44e8-8ecd-d0cd8297b43e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.594119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a98109-cecd-44e8-8ecd-d0cd8297b43e admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.594787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a98109-cecd-44e8-8ecd-d0cd8297b43e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:41.595092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a98109-cecd-44e8-8ecd-d0cd8297b43e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:41.596747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a98109-cecd-44e8-8ecd-d0cd8297b43e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:41.611683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 863/1726] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:41.616137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a98109-cecd-44e8-8ecd-d0cd8297b43e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:41.618092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a5a98109-cecd-44e8-8ecd-d0cd8297b43e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:41.633989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:41.721584 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:41.737885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 864/1727] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:41.740180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbce6b-f50e-4fe0-9ef5-a3605813f1d0 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:41.755603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbce6b-f50e-4fe0-9ef5-a3605813f1d0 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:41.764542 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34587e71-f25c-4206-bc02-716e7dd9672b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:41.765830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34587e71-f25c-4206-bc02-716e7dd9672b admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.766075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34587e71-f25c-4206-bc02-716e7dd9672b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.766355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34587e71-f25c-4206-bc02-716e7dd9672b admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.767645 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34587e71-f25c-4206-bc02-716e7dd9672b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:41.769807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34587e71-f25c-4206-bc02-716e7dd9672b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:41.800363 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a98109-cecd-44e8-8ecd-d0cd8297b43e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:41.815282 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1efc20-7b0e-4861-8fb7-fb0e29e253ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:41.816881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1efc20-7b0e-4861-8fb7-fb0e29e253ef service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.819520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1efc20-7b0e-4861-8fb7-fb0e29e253ef service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.820608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1efc20-7b0e-4861-8fb7-fb0e29e253ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.824135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 865/1728] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:41.828403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34587e71-f25c-4206-bc02-716e7dd9672b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:41.833513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-34587e71-f25c-4206-bc02-716e7dd9672b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:41.842412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1efc20-7b0e-4861-8fb7-fb0e29e253ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:41.843990 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1efc20-7b0e-4861-8fb7-fb0e29e253ef service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:41.883639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 864/1729] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:41.886106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1efc20-7b0e-4861-8fb7-fb0e29e253ef service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:41.941680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1efc20-7b0e-4861-8fb7-fb0e29e253ef service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:41.959349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa20e4-f9e8-4c25-92a2-ed6b95ae65d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:41.960000 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa20e4-f9e8-4c25-92a2-ed6b95ae65d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.960512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa20e4-f9e8-4c25-92a2-ed6b95ae65d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.960783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa20e4-f9e8-4c25-92a2-ed6b95ae65d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.961416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa20e4-f9e8-4c25-92a2-ed6b95ae65d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:41.963372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa20e4-f9e8-4c25-92a2-ed6b95ae65d6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:42.007776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34587e71-f25c-4206-bc02-716e7dd9672b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:42.017833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42cde2-15b5-4f73-8141-ab5ac861c345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:42.018846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42cde2-15b5-4f73-8141-ab5ac861c345 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:42.019325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42cde2-15b5-4f73-8141-ab5ac861c345 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:42.019805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42cde2-15b5-4f73-8141-ab5ac861c345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:42.075489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 865/1730] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:41 2022] POST /identity/v3/projects => generated 401 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:42.075489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa20e4-f9e8-4c25-92a2-ed6b95ae65d6 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:42.154029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42cde2-15b5-4f73-8141-ab5ac861c345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:42.155771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42cde2-15b5-4f73-8141-ab5ac861c345 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:42.187731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 866/1731] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:42.205862 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae10d5-9afb-46a5-925e-a44b74fc136d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:42.207324 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae10d5-9afb-46a5-925e-a44b74fc136d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:42.207633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae10d5-9afb-46a5-925e-a44b74fc136d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:42.277304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa20e4-f9e8-4c25-92a2-ed6b95ae65d6 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:42.301134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f009fa-d1ed-4bc5-b00f-89e353411c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:42.301911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f009fa-d1ed-4bc5-b00f-89e353411c8e admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:42.302190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f009fa-d1ed-4bc5-b00f-89e353411c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:42.302453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f009fa-d1ed-4bc5-b00f-89e353411c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:42.304289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f009fa-d1ed-4bc5-b00f-89e353411c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:42.308202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f009fa-d1ed-4bc5-b00f-89e353411c8e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:42.334050 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-78ae10d5-9afb-46a5-925e-a44b74fc136d None None] MFA Rules not processed for user `9f07158e4fd1430e839a1592e1e11b58`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:42.339473 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-78ae10d5-9afb-46a5-925e-a44b74fc136d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:42.381218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 866/1732] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:42 2022] POST /identity/v3/projects => generated 355 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:42.382702 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f009fa-d1ed-4bc5-b00f-89e353411c8e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:42.475646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:42.508024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 867/1733] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:42.509537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae10d5-9afb-46a5-925e-a44b74fc136d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:42.523255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae10d5-9afb-46a5-925e-a44b74fc136d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:42.647626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f009fa-d1ed-4bc5-b00f-89e353411c8e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:42.671960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d25bf3-e07d-46dc-82f3-5e349bfa8a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:42.673557 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d25bf3-e07d-46dc-82f3-5e349bfa8a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:42.674029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d25bf3-e07d-46dc-82f3-5e349bfa8a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:42.677324 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b7551-08fa-4210-aca7-80b4a0f6d5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:42.678084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b7551-08fa-4210-aca7-80b4a0f6d5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:42.678214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d25bf3-e07d-46dc-82f3-5e349bfa8a68 admin admin] PATH_INFO: `/v3/projects/72927f397b1c4a259dc1414e8f3aacd3/users/20a38313912c4b57a288cf11c57f498c/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:42.678375 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b7551-08fa-4210-aca7-80b4a0f6d5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:42.679023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b7551-08fa-4210-aca7-80b4a0f6d5d5 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:42.679427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b7551-08fa-4210-aca7-80b4a0f6d5d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:42.681861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56b7551-08fa-4210-aca7-80b4a0f6d5d5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:42.684504 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d25bf3-e07d-46dc-82f3-5e349bfa8a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72927f397b1c4a259dc1414e8f3aacd3, user_id=20a38313912c4b57a288cf11c57f498c, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:42.695800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d25bf3-e07d-46dc-82f3-5e349bfa8a68 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:42.721175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 867/1734] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:42 2022] PUT /identity/v3/projects/72927f397b1c4a259dc1414e8f3aacd3/users/20a38313912c4b57a288cf11c57f498c/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:42.721979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d25bf3-e07d-46dc-82f3-5e349bfa8a68 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:42.750839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 868/1735] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:42 2022] POST /identity/v3/users => generated 301 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:42.773044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe0cf6-7d69-43e2-b52a-55fdadb18383 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:42.773408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe0cf6-7d69-43e2-b52a-55fdadb18383 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:42.774966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe0cf6-7d69-43e2-b52a-55fdadb18383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:42.839243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8efe0cf6-7d69-43e2-b52a-55fdadb18383 None None] MFA Rules not processed for user `9f07158e4fd1430e839a1592e1e11b58`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:42.842345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:42.843660 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8efe0cf6-7d69-43e2-b52a-55fdadb18383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:42.922310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d25bf3-e07d-46dc-82f3-5e349bfa8a68 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:42.945226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da5cb6-f512-4b22-9916-4546c902bab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:42.949822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da5cb6-f512-4b22-9916-4546c902bab3 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:42.950420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da5cb6-f512-4b22-9916-4546c902bab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:42.950707 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da5cb6-f512-4b22-9916-4546c902bab3 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:42.951378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da5cb6-f512-4b22-9916-4546c902bab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:42.962020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da5cb6-f512-4b22-9916-4546c902bab3 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:43.039754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 868/1736] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:42 2022] POST /identity/v3/users => generated 284 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:43.043276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e015a2-af91-4650-8578-2bbc20f820b8 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:43.043563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e015a2-af91-4650-8578-2bbc20f820b8 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:43.043836 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e015a2-af91-4650-8578-2bbc20f820b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:43.084984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 869/1737] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:43.090236 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf23bc-9fde-4e3a-b6e8-ed6564232908 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:43.093579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf23bc-9fde-4e3a-b6e8-ed6564232908 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:43.096885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf23bc-9fde-4e3a-b6e8-ed6564232908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:43.114414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-d6e015a2-af91-4650-8578-2bbc20f820b8 None None] MFA Rules not processed for user `96744eb6a9cf4895b6101e209d1fdd29`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:43.114928 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d6e015a2-af91-4650-8578-2bbc20f820b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:43.207385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7fcf23bc-9fde-4e3a-b6e8-ed6564232908 None None] MFA Rules not processed for user `20a38313912c4b57a288cf11c57f498c`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:43.208637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7fcf23bc-9fde-4e3a-b6e8-ed6564232908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:43.243366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 869/1738] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:43.243366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e015a2-af91-4650-8578-2bbc20f820b8 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:43.267472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e015a2-af91-4650-8578-2bbc20f820b8 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:43.279012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60d895-592b-45b1-b15e-2058253c61a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:43.281955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60d895-592b-45b1-b15e-2058253c61a0 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:43.282263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60d895-592b-45b1-b15e-2058253c61a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:43.282546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60d895-592b-45b1-b15e-2058253c61a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:43.283320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60d895-592b-45b1-b15e-2058253c61a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:43.285264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60d895-592b-45b1-b15e-2058253c61a0 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:43.318856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 870/1739] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:43.328818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60d895-592b-45b1-b15e-2058253c61a0 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:43.391800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 870/1740] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:43.393325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf23bc-9fde-4e3a-b6e8-ed6564232908 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:43.412145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf23bc-9fde-4e3a-b6e8-ed6564232908 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:43.421407 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149c982-9b80-4e81-a3e4-94c69cea94ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:43.422311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149c982-9b80-4e81-a3e4-94c69cea94ff admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:43.422664 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149c982-9b80-4e81-a3e4-94c69cea94ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:43.423010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149c982-9b80-4e81-a3e4-94c69cea94ff admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:43.424110 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149c982-9b80-4e81-a3e4-94c69cea94ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:43.425998 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149c982-9b80-4e81-a3e4-94c69cea94ff admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:43.444204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 871/1741] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:43.444204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149c982-9b80-4e81-a3e4-94c69cea94ff admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:43.450754 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:43.490544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60d895-592b-45b1-b15e-2058253c61a0 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:43.508566 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bed0a-cd55-4417-8354-ae944f6423dc tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1596ff6299418db7ca7166795d435f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a7acd2e8ea543cba3094bf34c4d5af5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:43.509215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bed0a-cd55-4417-8354-ae944f6423dc tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:43.523921 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bed0a-cd55-4417-8354-ae944f6423dc tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:43.525220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bed0a-cd55-4417-8354-ae944f6423dc tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:43.527823 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bed0a-cd55-4417-8354-ae944f6423dc tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:43.528160 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bed0a-cd55-4417-8354-ae944f6423dc tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:43.531166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bed0a-cd55-4417-8354-ae944f6423dc tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:43.563587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 871/1742] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:43.566781 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4b84a-c2a5-42bd-b6e3-0f2c76e7d190 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:43.567073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4b84a-c2a5-42bd-b6e3-0f2c76e7d190 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:43.567348 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4b84a-c2a5-42bd-b6e3-0f2c76e7d190 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:43.568716 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 872/1743] 10.222.0.55 () {54 vars in 961 bytes} [Thu Sep 8 13:52:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:43.569466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4b84a-c2a5-42bd-b6e3-0f2c76e7d190 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:43.601487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4b84a-c2a5-42bd-b6e3-0f2c76e7d190 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:43.618018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a47048-e0e6-4dd0-94e5-a41f5a048a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:43.618018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a47048-e0e6-4dd0-94e5-a41f5a048a73 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:43.618018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a47048-e0e6-4dd0-94e5-a41f5a048a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:43.628280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a47048-e0e6-4dd0-94e5-a41f5a048a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:43.629062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a47048-e0e6-4dd0-94e5-a41f5a048a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:43.631928 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a47048-e0e6-4dd0-94e5-a41f5a048a73 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:43.642082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149c982-9b80-4e81-a3e4-94c69cea94ff admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:43.651572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a278fe9-8290-43d9-a335-3debb42240bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:43.652580 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a278fe9-8290-43d9-a335-3debb42240bc service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:43.653087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a278fe9-8290-43d9-a335-3debb42240bc service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:43.653570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a278fe9-8290-43d9-a335-3debb42240bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:43.663926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 873/1744] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:43.665518 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a47048-e0e6-4dd0-94e5-a41f5a048a73 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:43.669323 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-19a47048-e0e6-4dd0-94e5-a41f5a048a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:43.672242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a278fe9-8290-43d9-a335-3debb42240bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:43.674994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a278fe9-8290-43d9-a335-3debb42240bc service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:43.691074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 872/1745] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:43.692789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a278fe9-8290-43d9-a335-3debb42240bc service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:43.695439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a47048-e0e6-4dd0-94e5-a41f5a048a73 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:43.697211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2a278fe9-8290-43d9-a335-3debb42240bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:43.711532 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bdafb-123c-4655-98b8-c241805c303e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:43.713385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bdafb-123c-4655-98b8-c241805c303e service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:43.716836 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bdafb-123c-4655-98b8-c241805c303e service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:43.716836 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771bdafb-123c-4655-98b8-c241805c303e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:43.735152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bdafb-123c-4655-98b8-c241805c303e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:43.736760 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771bdafb-123c-4655-98b8-c241805c303e service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:43.769541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 874/1746] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:43.770500 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bdafb-123c-4655-98b8-c241805c303e service nova] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:43.799028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771bdafb-123c-4655-98b8-c241805c303e service nova] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:43.806556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9f0a0-90d0-4233-9763-404b7b4e9456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:43.807233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9f0a0-90d0-4233-9763-404b7b4e9456 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:43.807514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9f0a0-90d0-4233-9763-404b7b4e9456 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:43.807790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9f0a0-90d0-4233-9763-404b7b4e9456 admin admin] PATH_INFO: `/v3/projects/d88a346a96a94eb89843dbcf7664ca00/users/a88f709b48034fb3809d435869398b02/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:43.848709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9f0a0-90d0-4233-9763-404b7b4e9456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d88a346a96a94eb89843dbcf7664ca00, user_id=a88f709b48034fb3809d435869398b02, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:43.852188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9f0a0-90d0-4233-9763-404b7b4e9456 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:43.855302 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:43.866862 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a278fe9-8290-43d9-a335-3debb42240bc service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:43.876398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 875/1747] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:43 2022] PUT /identity/v3/projects/d88a346a96a94eb89843dbcf7664ca00/users/a88f709b48034fb3809d435869398b02/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:43.877764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9f0a0-90d0-4233-9763-404b7b4e9456 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:44.020967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9f0a0-90d0-4233-9763-404b7b4e9456 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:44.050393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca2556-170e-4ce6-9276-f292437803a9 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1596ff6299418db7ca7166795d435f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a7acd2e8ea543cba3094bf34c4d5af5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:44.051851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca2556-170e-4ce6-9276-f292437803a9 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:44.052653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca2556-170e-4ce6-9276-f292437803a9 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:44.052944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca2556-170e-4ce6-9276-f292437803a9 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:44.053651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca2556-170e-4ce6-9276-f292437803a9 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:44.055763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca2556-170e-4ce6-9276-f292437803a9 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:44.084324 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac65e15-3d80-445a-99b2-fbb8de3fbade service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:44.085466 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac65e15-3d80-445a-99b2-fbb8de3fbade service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:44.085839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac65e15-3d80-445a-99b2-fbb8de3fbade service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:44.086120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac65e15-3d80-445a-99b2-fbb8de3fbade service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:44.112199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 876/1748] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:43 2022] POST /identity/v3/projects => generated 355 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:44.120932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cf3d2-dd33-4e05-97b1-5ed5c8ca2981 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:44.121231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cf3d2-dd33-4e05-97b1-5ed5c8ca2981 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:44.121797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27cf3d2-dd33-4e05-97b1-5ed5c8ca2981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:44.176828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-a27cf3d2-dd33-4e05-97b1-5ed5c8ca2981 None None] MFA Rules not processed for user `e42fb05dbae64c758098c591d6ffa790`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:44.178188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a27cf3d2-dd33-4e05-97b1-5ed5c8ca2981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:44.243754 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:44.251026 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac65e15-3d80-445a-99b2-fbb8de3fbade service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:44.254713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac65e15-3d80-445a-99b2-fbb8de3fbade service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:44.271044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 873/1749] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:44.272168 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac65e15-3d80-445a-99b2-fbb8de3fbade service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:44.313761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 877/1750] 10.222.0.55 () {58 vars in 1058 bytes} [Thu Sep 8 13:52:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:44.314739 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cf3d2-dd33-4e05-97b1-5ed5c8ca2981 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:44.451221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac65e15-3d80-445a-99b2-fbb8de3fbade service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:44.507044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27cf3d2-dd33-4e05-97b1-5ed5c8ca2981 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:44.522392 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047cf1c-23cb-4a8b-9c96-648bafe3d9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:44.523089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047cf1c-23cb-4a8b-9c96-648bafe3d9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:44.523434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047cf1c-23cb-4a8b-9c96-648bafe3d9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:44.523878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e047cf1c-23cb-4a8b-9c96-648bafe3d9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:44.524701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047cf1c-23cb-4a8b-9c96-648bafe3d9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:44.526474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e047cf1c-23cb-4a8b-9c96-648bafe3d9d0 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:44.575207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 878/1751] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:44.577932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047cf1c-23cb-4a8b-9c96-648bafe3d9d0 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:44.596392 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ce806-64df-40d9-9cc7-18e2bbb5eb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:44.597267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ce806-64df-40d9-9cc7-18e2bbb5eb05 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:44.597614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ce806-64df-40d9-9cc7-18e2bbb5eb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:44.597983 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ce806-64df-40d9-9cc7-18e2bbb5eb05 admin admin] PATH_INFO: `/v3/projects/3634641d25f749c5aeb08602d75a9ed3/users/6c9448a5b0234892a84e4d930a066297/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:44.609682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:44.633970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ce806-64df-40d9-9cc7-18e2bbb5eb05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3634641d25f749c5aeb08602d75a9ed3, user_id=6c9448a5b0234892a84e4d930a066297, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:44.636455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ce806-64df-40d9-9cc7-18e2bbb5eb05 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:44.670433 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 874/1752] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:44 2022] PUT /identity/v3/projects/3634641d25f749c5aeb08602d75a9ed3/users/6c9448a5b0234892a84e4d930a066297/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:44.670433 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ce806-64df-40d9-9cc7-18e2bbb5eb05 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:44.735797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e047cf1c-23cb-4a8b-9c96-648bafe3d9d0 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:44.837591 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ce806-64df-40d9-9cc7-18e2bbb5eb05 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:44.849834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0f022-56ed-4e86-93d7-1766980d56e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:44.854260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0f022-56ed-4e86-93d7-1766980d56e8 service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:44.857289 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0f022-56ed-4e86-93d7-1766980d56e8 service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:44.865788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0f022-56ed-4e86-93d7-1766980d56e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:44.870285 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ffa0f022-56ed-4e86-93d7-1766980d56e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:44.906884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0f022-56ed-4e86-93d7-1766980d56e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:44.908468 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0f022-56ed-4e86-93d7-1766980d56e8 service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:44.956365 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 875/1753] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:44.958447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e86c6-3096-47e0-9f37-9cbf717be16d None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:44.958619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e86c6-3096-47e0-9f37-9cbf717be16d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:44.958896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e86c6-3096-47e0-9f37-9cbf717be16d None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:44.987859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b7b81e-19fc-4c95-9fd7-7cb7e61925bf tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1596ff6299418db7ca7166795d435f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a7acd2e8ea543cba3094bf34c4d5af5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:44.988641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7b81e-19fc-4c95-9fd7-7cb7e61925bf tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:44.988915 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7b81e-19fc-4c95-9fd7-7cb7e61925bf tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:44.989197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b7b81e-19fc-4c95-9fd7-7cb7e61925bf tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:44.991142 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 876/1754] 10.222.0.55 () {54 vars in 912 bytes} [Thu Sep 8 13:52:44 2022] GET /identity => generated 270 bytes in 34 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:44.992231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e86c6-3096-47e0-9f37-9cbf717be16d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:44.993889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7b81e-19fc-4c95-9fd7-7cb7e61925bf tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:44.994260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7b81e-19fc-4c95-9fd7-7cb7e61925bf tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:44.999869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b7b81e-19fc-4c95-9fd7-7cb7e61925bf tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:45.038210 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 879/1755] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:45.039502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4013d5-824a-4e55-bbcd-24581177b83a None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:45.039777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4013d5-824a-4e55-bbcd-24581177b83a None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:45.040053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4013d5-824a-4e55-bbcd-24581177b83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:45.044479 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:45.094609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-9b4013d5-824a-4e55-bbcd-24581177b83a None None] MFA Rules not processed for user `6c9448a5b0234892a84e4d930a066297`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:45.095848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-9b4013d5-824a-4e55-bbcd-24581177b83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:45.177646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e86c6-3096-47e0-9f37-9cbf717be16d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:45.188065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876f07d-895c-4e0f-8854-4754446363a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:45.189069 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876f07d-895c-4e0f-8854-4754446363a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:45.190541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876f07d-895c-4e0f-8854-4754446363a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:45.191517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876f07d-895c-4e0f-8854-4754446363a0 admin admin] PATH_INFO: `/v3/projects/d88a346a96a94eb89843dbcf7664ca00/users/a88f709b48034fb3809d435869398b02/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:45.197309 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876f07d-895c-4e0f-8854-4754446363a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d88a346a96a94eb89843dbcf7664ca00, user_id=a88f709b48034fb3809d435869398b02, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:45.201908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876f07d-895c-4e0f-8854-4754446363a0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:45.288953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 877/1756] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:44 2022] PUT /identity/v3/projects/d88a346a96a94eb89843dbcf7664ca00/users/a88f709b48034fb3809d435869398b02/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:45.297060 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f7ee-8093-412d-af26-bb85bcd5c9e3 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:45.297403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f7ee-8093-412d-af26-bb85bcd5c9e3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:45.297686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f7ee-8093-412d-af26-bb85bcd5c9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:45.310175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 880/1757] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:45 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:45.312713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f06458-8c61-430b-b031-4a210430e5d8 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:45.314028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f06458-8c61-430b-b031-4a210430e5d8 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:45.315545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f06458-8c61-430b-b031-4a210430e5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:45.329430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5205f7ee-8093-412d-af26-bb85bcd5c9e3 None None] MFA Rules not processed for user `2e48a8c1b91142cf9d1af17f28e41961`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:45.330709 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5205f7ee-8093-412d-af26-bb85bcd5c9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:45.357848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-06f06458-8c61-430b-b031-4a210430e5d8 None None] MFA Rules not processed for user `531ebc79bf424e9b8762b2c7ec7fe8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:45.358250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-06f06458-8c61-430b-b031-4a210430e5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:45.459078 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:45.487052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 878/1758] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:45.487901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205f7ee-8093-412d-af26-bb85bcd5c9e3 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:45.496259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 881/1759] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 13:52:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:45.497159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f06458-8c61-430b-b031-4a210430e5d8 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:45.625578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f06458-8c61-430b-b031-4a210430e5d8 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:45.630969 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b9438-eb8e-4532-829e-05cb091e5759 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1596ff6299418db7ca7166795d435f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a7acd2e8ea543cba3094bf34c4d5af5', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:45.631693 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b9438-eb8e-4532-829e-05cb091e5759 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:45.632187 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b9438-eb8e-4532-829e-05cb091e5759 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:45.632538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b9438-eb8e-4532-829e-05cb091e5759 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] PATH_INFO: `/v3/projects/efc5b3585c2d49afb71d1162e0519c0d` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:45.633828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b9438-eb8e-4532-829e-05cb091e5759 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: Authorizing `identity:delete_project(project_id=efc5b3585c2d49afb71d1162e0519c0d)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:45.636703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b9438-eb8e-4532-829e-05cb091e5759 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:45.703211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205f7ee-8093-412d-af26-bb85bcd5c9e3 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:45.716932 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:45.717603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:45.718452 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:45.718727 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] PATH_INFO: `/v3/users/b4b7305e55ce4d62a4a2fcb8882c0f39` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:45.723849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4b7305e55ce4d62a4a2fcb8882c0f39)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:45.731692 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8e6b9438-eb8e-4532-829e-05cb091e5759 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc5b3585c2d49afb71d1162e0519c0d', 'request_id': 'req-8e6b9438-eb8e-4532-829e-05cb091e5759'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:45.735528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:45.746816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8e6b9438-eb8e-4532-829e-05cb091e5759 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc5b3585c2d49afb71d1162e0519c0d', 'request_id': 'req-8e6b9438-eb8e-4532-829e-05cb091e5759'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:45.762393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8e6b9438-eb8e-4532-829e-05cb091e5759 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc5b3585c2d49afb71d1162e0519c0d', 'request_id': 'req-8e6b9438-eb8e-4532-829e-05cb091e5759'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:45.783287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8e6b9438-eb8e-4532-829e-05cb091e5759 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] The token cache is being invalidate because project efc5b3585c2d49afb71d1162e0519c0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:45.783684 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8e6b9438-eb8e-4532-829e-05cb091e5759 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:45.784440 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8e6b9438-eb8e-4532-829e-05cb091e5759 tempest-VolumeQuotaClassesTest-459760735 tempest-VolumeQuotaClassesTest-459760735-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:45.790260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 882/1760] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:45 2022] DELETE /identity/v3/projects/efc5b3585c2d49afb71d1162e0519c0d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:45.791945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e2a61-d0b8-4125-a05f-b86a6bd152ac None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:45.792030 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e2a61-d0b8-4125-a05f-b86a6bd152ac None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:45.792605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e2a61-d0b8-4125-a05f-b86a6bd152ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:45.832954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-c00e2a61-d0b8-4125-a05f-b86a6bd152ac None None] MFA Rules not processed for user `a88f709b48034fb3809d435869398b02`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:45.834359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c00e2a61-d0b8-4125-a05f-b86a6bd152ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:45.874970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4b7305e55ce4d62a4a2fcb8882c0f39'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:45.892574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4b7305e55ce4d62a4a2fcb8882c0f39'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:45.893817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4b7305e55ce4d62a4a2fcb8882c0f39'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:45.896528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4b7305e55ce4d62a4a2fcb8882c0f39'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:45.900906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:45.912529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4b7305e55ce4d62a4a2fcb8882c0f39'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:45.964578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 879/1761] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:52:45 2022] DELETE /identity/v3/users/b4b7305e55ce4d62a4a2fcb8882c0f39 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:45.965469 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:45.972284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:46.004159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 883/1762] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:46.006687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e2a61-d0b8-4125-a05f-b86a6bd152ac None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:46.012372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c00e2a61-d0b8-4125-a05f-b86a6bd152ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:46.063887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:46.127652 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc5213-2145-4d03-bf33-930fd66eccd4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:46.146400 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1521f42-bb8d-4449-ab0e-f5df3581766f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:46.146910 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1521f42-bb8d-4449-ab0e-f5df3581766f admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:46.147199 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1521f42-bb8d-4449-ab0e-f5df3581766f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:46.147477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1521f42-bb8d-4449-ab0e-f5df3581766f admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:46.148136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1521f42-bb8d-4449-ab0e-f5df3581766f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:46.150747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1521f42-bb8d-4449-ab0e-f5df3581766f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:46.223949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 880/1763] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:45 2022] POST /identity/v3/projects => generated 405 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:46.226328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e8f89-b37f-4f42-a21a-82884b5f12c4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:46.226895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e8f89-b37f-4f42-a21a-82884b5f12c4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:46.227046 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e8f89-b37f-4f42-a21a-82884b5f12c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:46.229574 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e2a61-d0b8-4125-a05f-b86a6bd152ac None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:46.253599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-286e8f89-b37f-4f42-a21a-82884b5f12c4 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:46.255960 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-286e8f89-b37f-4f42-a21a-82884b5f12c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:46.372049 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294e3ee-10b5-4693-9975-bf412d6cbd71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:46.380641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294e3ee-10b5-4693-9975-bf412d6cbd71 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:46.381890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294e3ee-10b5-4693-9975-bf412d6cbd71 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:46.383134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294e3ee-10b5-4693-9975-bf412d6cbd71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:46.384162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 881/1764] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:46.385233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e8f89-b37f-4f42-a21a-82884b5f12c4 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:46.386744 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-286e8f89-b37f-4f42-a21a-82884b5f12c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:46.388881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1294e3ee-10b5-4693-9975-bf412d6cbd71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:46.415270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294e3ee-10b5-4693-9975-bf412d6cbd71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:46.419694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294e3ee-10b5-4693-9975-bf412d6cbd71 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:46.449265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 884/1765] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:46.450830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294e3ee-10b5-4693-9975-bf412d6cbd71 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:46.478954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294e3ee-10b5-4693-9975-bf412d6cbd71 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:46.490554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331e4ad-63bb-4b84-99b4-8f5d8901685e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:46.491519 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331e4ad-63bb-4b84-99b4-8f5d8901685e admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:46.492153 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331e4ad-63bb-4b84-99b4-8f5d8901685e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:46.492927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331e4ad-63bb-4b84-99b4-8f5d8901685e admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:46.494556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331e4ad-63bb-4b84-99b4-8f5d8901685e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:46.500683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331e4ad-63bb-4b84-99b4-8f5d8901685e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:46.531558 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 885/1766] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:46 2022] POST /identity/v3/users => generated 309 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:46.537242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e8f89-b37f-4f42-a21a-82884b5f12c4 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:46.537474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331e4ad-63bb-4b84-99b4-8f5d8901685e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:46.541129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a331e4ad-63bb-4b84-99b4-8f5d8901685e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:46.552448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008237fc-bb0b-42aa-8252-3ec2649cf667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:46.553101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008237fc-bb0b-42aa-8252-3ec2649cf667 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:46.553593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008237fc-bb0b-42aa-8252-3ec2649cf667 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:46.553707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008237fc-bb0b-42aa-8252-3ec2649cf667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:46.579235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:46.605393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331e4ad-63bb-4b84-99b4-8f5d8901685e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:46.619159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc338b-58a3-4ac2-8bfd-e860b42adaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:46.620810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc338b-58a3-4ac2-8bfd-e860b42adaf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:46.621337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc338b-58a3-4ac2-8bfd-e860b42adaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:46.621839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc338b-58a3-4ac2-8bfd-e860b42adaf0 admin admin] PATH_INFO: `/v3/users/bb44795a0a8b4f8189e82b21826c96b2` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:46.624968 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc338b-58a3-4ac2-8bfd-e860b42adaf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb44795a0a8b4f8189e82b21826c96b2)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:46.632310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc338b-58a3-4ac2-8bfd-e860b42adaf0 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:46.707795 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008237fc-bb0b-42aa-8252-3ec2649cf667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:46.709264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008237fc-bb0b-42aa-8252-3ec2649cf667 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:46.724470 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 882/1767] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:46.725673 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008237fc-bb0b-42aa-8252-3ec2649cf667 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:46.739877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008237fc-bb0b-42aa-8252-3ec2649cf667 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:46.748329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b159210-729d-4973-9876-3aa3f2de2a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:46.749166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b159210-729d-4973-9876-3aa3f2de2a5f service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:46.749993 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b159210-729d-4973-9876-3aa3f2de2a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:46.750768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b159210-729d-4973-9876-3aa3f2de2a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:46.767176 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b159210-729d-4973-9876-3aa3f2de2a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:46.768704 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b159210-729d-4973-9876-3aa3f2de2a5f service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:46.771981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3bcc338b-58a3-4ac2-8bfd-e860b42adaf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb44795a0a8b4f8189e82b21826c96b2'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:46.789882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3bcc338b-58a3-4ac2-8bfd-e860b42adaf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb44795a0a8b4f8189e82b21826c96b2'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:46.790982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3bcc338b-58a3-4ac2-8bfd-e860b42adaf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb44795a0a8b4f8189e82b21826c96b2'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:46.791718 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3bcc338b-58a3-4ac2-8bfd-e860b42adaf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb44795a0a8b4f8189e82b21826c96b2'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:46.813917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3bcc338b-58a3-4ac2-8bfd-e860b42adaf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb44795a0a8b4f8189e82b21826c96b2'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:46.845113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 883/1768] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:46.849677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b159210-729d-4973-9876-3aa3f2de2a5f service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:46.854406 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 886/1769] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:52:46 2022] DELETE /identity/v3/users/bb44795a0a8b4f8189e82b21826c96b2 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:46.862502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c034b67-c653-4c1c-965b-6b7f581c5b6d None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:46.864695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c034b67-c653-4c1c-965b-6b7f581c5b6d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:46.865138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c034b67-c653-4c1c-965b-6b7f581c5b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:46.880150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b159210-729d-4973-9876-3aa3f2de2a5f service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:46.935956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-2c034b67-c653-4c1c-965b-6b7f581c5b6d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:46.938057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2c034b67-c653-4c1c-965b-6b7f581c5b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:46.963035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:47.031910 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221e9c9-332a-4efc-9b7f-62c446cdd500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:47.032633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221e9c9-332a-4efc-9b7f-62c446cdd500 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:47.032975 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221e9c9-332a-4efc-9b7f-62c446cdd500 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:47.033316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9221e9c9-332a-4efc-9b7f-62c446cdd500 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:47.034290 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221e9c9-332a-4efc-9b7f-62c446cdd500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:47.047711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9221e9c9-332a-4efc-9b7f-62c446cdd500 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:47.081711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 884/1770] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:47.082915 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221e9c9-332a-4efc-9b7f-62c446cdd500 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:47.089963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9221e9c9-332a-4efc-9b7f-62c446cdd500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:47.092629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 887/1771] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:47.096438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c034b67-c653-4c1c-965b-6b7f581c5b6d None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:47.100086 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2c034b67-c653-4c1c-965b-6b7f581c5b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:47.143287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c034b67-c653-4c1c-965b-6b7f581c5b6d None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:47.160004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ec993-922d-4571-af5f-cdb9bd704baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:47.160694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ec993-922d-4571-af5f-cdb9bd704baa admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:47.160984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ec993-922d-4571-af5f-cdb9bd704baa admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:47.163780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ec993-922d-4571-af5f-cdb9bd704baa admin admin] PATH_INFO: `/v3/projects/6dee55a1c3454a8baf3346e688416405/users/77c4a80c845140d497c69369abd0bef5/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:47.208515 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ec993-922d-4571-af5f-cdb9bd704baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dee55a1c3454a8baf3346e688416405, user_id=77c4a80c845140d497c69369abd0bef5, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:47.213179 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9ec993-922d-4571-af5f-cdb9bd704baa admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:47.232294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9221e9c9-332a-4efc-9b7f-62c446cdd500 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:47.238659 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 888/1772] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:47 2022] PUT /identity/v3/projects/6dee55a1c3454a8baf3346e688416405/users/77c4a80c845140d497c69369abd0bef5/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:47.240460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ec993-922d-4571-af5f-cdb9bd704baa admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:47.243778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8f9ec993-922d-4571-af5f-cdb9bd704baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:47.333332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:47.347959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84c20a-9aee-4a8a-aa1c-81c0c999197f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:47.348585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84c20a-9aee-4a8a-aa1c-81c0c999197f service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:47.348869 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84c20a-9aee-4a8a-aa1c-81c0c999197f service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:47.349143 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84c20a-9aee-4a8a-aa1c-81c0c999197f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:47.354034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1c84c20a-9aee-4a8a-aa1c-81c0c999197f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:47.404529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9ec993-922d-4571-af5f-cdb9bd704baa admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:47.426321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68f343-0d0c-4451-b2b1-4a267065d172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:47.430247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68f343-0d0c-4451-b2b1-4a267065d172 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:47.430247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68f343-0d0c-4451-b2b1-4a267065d172 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:47.430453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68f343-0d0c-4451-b2b1-4a267065d172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:47.508223 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84c20a-9aee-4a8a-aa1c-81c0c999197f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:47.509686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c84c20a-9aee-4a8a-aa1c-81c0c999197f service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:47.537421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 885/1773] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:47.548146 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84c20a-9aee-4a8a-aa1c-81c0c999197f service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:47.687834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68f343-0d0c-4451-b2b1-4a267065d172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:47.690481 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be68f343-0d0c-4451-b2b1-4a267065d172 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:47.739979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 889/1774] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:47.781049 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e9da2-6c5b-4145-82a3-67f26f3dbfa9 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:47.781680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e9da2-6c5b-4145-82a3-67f26f3dbfa9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:47.781680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e9da2-6c5b-4145-82a3-67f26f3dbfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:47.790974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84c20a-9aee-4a8a-aa1c-81c0c999197f service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:47.800408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67513a7-2f65-4ff2-8a82-32e74a153dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:47.802144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67513a7-2f65-4ff2-8a82-32e74a153dfa admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:47.802144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67513a7-2f65-4ff2-8a82-32e74a153dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:47.802671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67513a7-2f65-4ff2-8a82-32e74a153dfa admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:47.803908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67513a7-2f65-4ff2-8a82-32e74a153dfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:47.805460 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67513a7-2f65-4ff2-8a82-32e74a153dfa admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:47.821896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 886/1775] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:47.822916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67513a7-2f65-4ff2-8a82-32e74a153dfa admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:47.835995 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67513a7-2f65-4ff2-8a82-32e74a153dfa admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:47.862356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253777c9-968d-4133-bd24-421c735a902a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:47.862356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253777c9-968d-4133-bd24-421c735a902a admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:47.862356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253777c9-968d-4133-bd24-421c735a902a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:47.862356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253777c9-968d-4133-bd24-421c735a902a admin admin] PATH_INFO: `/v3/projects/6dee55a1c3454a8baf3346e688416405/users/77c4a80c845140d497c69369abd0bef5/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:47.864680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253777c9-968d-4133-bd24-421c735a902a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dee55a1c3454a8baf3346e688416405, user_id=77c4a80c845140d497c69369abd0bef5, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:47.876296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253777c9-968d-4133-bd24-421c735a902a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:47.916599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 887/1776] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:47 2022] PUT /identity/v3/projects/6dee55a1c3454a8baf3346e688416405/users/77c4a80c845140d497c69369abd0bef5/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:47.918981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eef798-626e-48b1-aeb4-e802f546cccd None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:47.923424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eef798-626e-48b1-aeb4-e802f546cccd None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:47.924035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eef798-626e-48b1-aeb4-e802f546cccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:47.925898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-098e9da2-6c5b-4145-82a3-67f26f3dbfa9 None None] MFA Rules not processed for user `96744eb6a9cf4895b6101e209d1fdd29`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:47.929783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-098e9da2-6c5b-4145-82a3-67f26f3dbfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:48.009969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-45eef798-626e-48b1-aeb4-e802f546cccd None None] MFA Rules not processed for user `77c4a80c845140d497c69369abd0bef5`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:48.013346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-45eef798-626e-48b1-aeb4-e802f546cccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:48.133870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 888/1777] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:47 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:48.159401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eef798-626e-48b1-aeb4-e802f546cccd None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:48.173019 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:48.185265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:48.232917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 890/1778] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:48.367606 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eef798-626e-48b1-aeb4-e802f546cccd None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:48.385272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab448d4-4d6d-4617-8ebd-73b8b563a560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:48.385931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab448d4-4d6d-4617-8ebd-73b8b563a560 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:48.386218 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab448d4-4d6d-4617-8ebd-73b8b563a560 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:48.386497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab448d4-4d6d-4617-8ebd-73b8b563a560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:48.422387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab448d4-4d6d-4617-8ebd-73b8b563a560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:48.429381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab448d4-4d6d-4617-8ebd-73b8b563a560 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:48.471389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e9da2-6c5b-4145-82a3-67f26f3dbfa9 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:48.473467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-098e9da2-6c5b-4145-82a3-67f26f3dbfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:48.490935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 889/1779] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:48 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:52:48.615675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e9da2-6c5b-4145-82a3-67f26f3dbfa9 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:48.622158 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:48.623201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:48.624260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:48.624856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] PATH_INFO: `/v3/projects/c5084e7f21bb43479da70ba3ae82f89f` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:48.630400 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5084e7f21bb43479da70ba3ae82f89f)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:48.632807 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:48.743841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5084e7f21bb43479da70ba3ae82f89f', 'request_id': 'req-1cd9cedb-bf51-42d0-97e1-0d608138af8f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:48.749721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786fc0a-2f09-413e-b829-15022d81e9a7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:48.750117 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786fc0a-2f09-413e-b829-15022d81e9a7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:48.750464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786fc0a-2f09-413e-b829-15022d81e9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:48.752299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5084e7f21bb43479da70ba3ae82f89f', 'request_id': 'req-1cd9cedb-bf51-42d0-97e1-0d608138af8f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:48.762312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5084e7f21bb43479da70ba3ae82f89f', 'request_id': 'req-1cd9cedb-bf51-42d0-97e1-0d608138af8f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:48.787597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] The token cache is being invalidate because project c5084e7f21bb43479da70ba3ae82f89f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:48.788212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:48.789268 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:48.799908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 891/1780] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:48 2022] DELETE /identity/v3/projects/c5084e7f21bb43479da70ba3ae82f89f => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:52:48.799908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:48.801851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:48.822940 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0786fc0a-2f09-413e-b829-15022d81e9a7 None None] MFA Rules not processed for user `9f07158e4fd1430e839a1592e1e11b58`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:48.824180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0786fc0a-2f09-413e-b829-15022d81e9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:48.974004 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 890/1781] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:48.976854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ff8de-f479-472b-a162-77e50c0f7f06 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:48.977335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ff8de-f479-472b-a162-77e50c0f7f06 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:48.977800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ff8de-f479-472b-a162-77e50c0f7f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:49.003907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9cedb-bf51-42d0-97e1-0d608138af8f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:49.010086 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:49.010495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:49.010839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:49.011176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] PATH_INFO: `/v3/users/20a38313912c4b57a288cf11c57f498c` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:49.032138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=20a38313912c4b57a288cf11c57f498c)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:49.036455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:49.091732 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:49.169385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-790ff8de-f479-472b-a162-77e50c0f7f06 None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:49.176183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-790ff8de-f479-472b-a162-77e50c0f7f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:49.190766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20a38313912c4b57a288cf11c57f498c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.219458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20a38313912c4b57a288cf11c57f498c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.222085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20a38313912c4b57a288cf11c57f498c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.226320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20a38313912c4b57a288cf11c57f498c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.260008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20a38313912c4b57a288cf11c57f498c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.317855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 892/1782] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:52:48 2022] DELETE /identity/v3/users/20a38313912c4b57a288cf11c57f498c => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:49.320762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:49.323635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:49.434698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 891/1783] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 13:52:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:49.437953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ff8de-f479-472b-a162-77e50c0f7f06 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:49.440097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-790ff8de-f479-472b-a162-77e50c0f7f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:49.480883 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ff8de-f479-472b-a162-77e50c0f7f06 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:49.516007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d6801-f6bb-4d4f-bbf9-2f5d5c9a3e2d admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:49.540130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:49.542022 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:49.542317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:49.542592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] PATH_INFO: `/v3/users/6c9448a5b0234892a84e4d930a066297` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:49.560406 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c9448a5b0234892a84e4d930a066297)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:49.565457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:49.576345 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:49.707374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c9448a5b0234892a84e4d930a066297'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.720962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c9448a5b0234892a84e4d930a066297'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.721771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c9448a5b0234892a84e4d930a066297'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.722441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c9448a5b0234892a84e4d930a066297'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.731025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088f008-f8da-49b2-bece-9d8293129bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:49.731833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088f008-f8da-49b2-bece-9d8293129bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:49.736910 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088f008-f8da-49b2-bece-9d8293129bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:49.737324 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088f008-f8da-49b2-bece-9d8293129bd1 admin admin] PATH_INFO: `/v3/projects/ef2a5c29fabc4219809253a7cfb7b26a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:49.737455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c9448a5b0234892a84e4d930a066297'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.740895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088f008-f8da-49b2-bece-9d8293129bd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2a5c29fabc4219809253a7cfb7b26a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:49.756087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088f008-f8da-49b2-bece-9d8293129bd1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:49.832988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 893/1784] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:52:49 2022] DELETE /identity/v3/users/6c9448a5b0234892a84e4d930a066297 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:49.834181 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:49.835961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:49.923656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:49.927047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6088f008-f8da-49b2-bece-9d8293129bd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2a5c29fabc4219809253a7cfb7b26a', 'request_id': 'req-6088f008-f8da-49b2-bece-9d8293129bd1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.943697 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6088f008-f8da-49b2-bece-9d8293129bd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2a5c29fabc4219809253a7cfb7b26a', 'request_id': 'req-6088f008-f8da-49b2-bece-9d8293129bd1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.960765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6088f008-f8da-49b2-bece-9d8293129bd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2a5c29fabc4219809253a7cfb7b26a', 'request_id': 'req-6088f008-f8da-49b2-bece-9d8293129bd1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.984606 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6088f008-f8da-49b2-bece-9d8293129bd1 admin admin] The token cache is being invalidate because project ef2a5c29fabc4219809253a7cfb7b26a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:49.985223 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6088f008-f8da-49b2-bece-9d8293129bd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.985917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6088f008-f8da-49b2-bece-9d8293129bd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:49.990573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 892/1785] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:49 2022] DELETE /identity/v3/projects/ef2a5c29fabc4219809253a7cfb7b26a => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:49.990573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadc8ca-a357-4137-b407-a8ec7908d29b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:49.990573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadc8ca-a357-4137-b407-a8ec7908d29b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:49.990573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadc8ca-a357-4137-b407-a8ec7908d29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:50.012456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-bdadc8ca-a357-4137-b407-a8ec7908d29b None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:50.014422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bdadc8ca-a357-4137-b407-a8ec7908d29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:50.106007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312648e-aff5-4e47-81e7-1668e0fe981e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:50.119527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 893/1786] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:50.121187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadc8ca-a357-4137-b407-a8ec7908d29b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:50.124657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bdadc8ca-a357-4137-b407-a8ec7908d29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:50.149528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadc8ca-a357-4137-b407-a8ec7908d29b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:50.157445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:50.158402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:50.159180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:50.159180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] PATH_INFO: `/v3/users/aa1596ff6299418db7ca7166795d435f` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:50.160532 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa1596ff6299418db7ca7166795d435f)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:50.172138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:50.219387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10900d80-62d9-43ab-9157-0bfe916bdcb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:50.220220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10900d80-62d9-43ab-9157-0bfe916bdcb8 service swift] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:50.220566 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10900d80-62d9-43ab-9157-0bfe916bdcb8 service swift] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:50.220966 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10900d80-62d9-43ab-9157-0bfe916bdcb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:50.223461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-10900d80-62d9-43ab-9157-0bfe916bdcb8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:50.245086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa1596ff6299418db7ca7166795d435f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:50.259654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa1596ff6299418db7ca7166795d435f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:50.261102 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa1596ff6299418db7ca7166795d435f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:50.261771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa1596ff6299418db7ca7166795d435f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:50.263760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:50.274583 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa1596ff6299418db7ca7166795d435f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:50.321600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 894/1787] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:52:50 2022] DELETE /identity/v3/users/aa1596ff6299418db7ca7166795d435f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:50.322941 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:50.324461 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:50.335907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10900d80-62d9-43ab-9157-0bfe916bdcb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:50.338355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10900d80-62d9-43ab-9157-0bfe916bdcb8 service swift] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:50.352550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 894/1788] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 13:52:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:52:50.353952 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10900d80-62d9-43ab-9157-0bfe916bdcb8 service swift] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:50.355490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-10900d80-62d9-43ab-9157-0bfe916bdcb8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:50.372879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10900d80-62d9-43ab-9157-0bfe916bdcb8 service swift] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:50.512262 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:50.549111 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4633dd8c-9bdc-4418-9046-e608fe91c778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:50.550671 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633dd8c-9bdc-4418-9046-e608fe91c778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:50.551053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633dd8c-9bdc-4418-9046-e608fe91c778 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:50.551564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4633dd8c-9bdc-4418-9046-e608fe91c778 admin admin] PATH_INFO: `/v3/projects/72927f397b1c4a259dc1414e8f3aacd3` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:50.560424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633dd8c-9bdc-4418-9046-e608fe91c778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72927f397b1c4a259dc1414e8f3aacd3)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:50.568845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4633dd8c-9bdc-4418-9046-e608fe91c778 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:50.576869 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c93977-00e2-41d3-901e-719f0915687b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:50.598085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b88d7e-2e46-4ca4-97de-45c98ab92d22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:50.602483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b88d7e-2e46-4ca4-97de-45c98ab92d22 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:50.602786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b88d7e-2e46-4ca4-97de-45c98ab92d22 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:50.603072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b88d7e-2e46-4ca4-97de-45c98ab92d22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:50.613129 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f7b88d7e-2e46-4ca4-97de-45c98ab92d22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:50.661805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b88d7e-2e46-4ca4-97de-45c98ab92d22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:50.676899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b88d7e-2e46-4ca4-97de-45c98ab92d22 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:50.712390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 895/1789] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:50.759503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4633dd8c-9bdc-4418-9046-e608fe91c778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72927f397b1c4a259dc1414e8f3aacd3', 'request_id': 'req-4633dd8c-9bdc-4418-9046-e608fe91c778'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:50.788354 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4633dd8c-9bdc-4418-9046-e608fe91c778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72927f397b1c4a259dc1414e8f3aacd3', 'request_id': 'req-4633dd8c-9bdc-4418-9046-e608fe91c778'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:50.812662 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4633dd8c-9bdc-4418-9046-e608fe91c778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72927f397b1c4a259dc1414e8f3aacd3', 'request_id': 'req-4633dd8c-9bdc-4418-9046-e608fe91c778'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:50.863196 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4633dd8c-9bdc-4418-9046-e608fe91c778 admin admin] The token cache is being invalidate because project 72927f397b1c4a259dc1414e8f3aacd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:50.863912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4633dd8c-9bdc-4418-9046-e608fe91c778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:50.865025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4633dd8c-9bdc-4418-9046-e608fe91c778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:50.871741 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 895/1790] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:50 2022] DELETE /identity/v3/projects/72927f397b1c4a259dc1414e8f3aacd3 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:50.879830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fe9bf-66d9-42d9-bee2-3f0d18e66ae2 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:50.880038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fe9bf-66d9-42d9-bee2-3f0d18e66ae2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:50.880415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fe9bf-66d9-42d9-bee2-3f0d18e66ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:50.942510 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-231fe9bf-66d9-42d9-bee2-3f0d18e66ae2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:50.944152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-231fe9bf-66d9-42d9-bee2-3f0d18e66ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:51.145020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 896/1791] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:51.153956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922a149-b62d-4247-b320-530126f5871c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:51.154398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922a149-b62d-4247-b320-530126f5871c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:51.154772 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922a149-b62d-4247-b320-530126f5871c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:51.193369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-c922a149-b62d-4247-b320-530126f5871c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:51.195556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c922a149-b62d-4247-b320-530126f5871c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:51.216186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 896/1792] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:51.220387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fe9bf-66d9-42d9-bee2-3f0d18e66ae2 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:51.241250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fe9bf-66d9-42d9-bee2-3f0d18e66ae2 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:51.256679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da071262-e955-4750-b483-8720e95cdd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:51.257530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da071262-e955-4750-b483-8720e95cdd25 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:51.257926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da071262-e955-4750-b483-8720e95cdd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:51.258400 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da071262-e955-4750-b483-8720e95cdd25 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:51.261846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da071262-e955-4750-b483-8720e95cdd25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:51.264181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da071262-e955-4750-b483-8720e95cdd25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:51.267481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da071262-e955-4750-b483-8720e95cdd25 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:51.310864 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 897/1793] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:52:51.316096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922a149-b62d-4247-b320-530126f5871c None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:51.333363 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922a149-b62d-4247-b320-530126f5871c None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:51.356009 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1260f24-6640-4bf3-b872-bf58187b91fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:51.358201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1260f24-6640-4bf3-b872-bf58187b91fb admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:51.361201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1260f24-6640-4bf3-b872-bf58187b91fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:51.361608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1260f24-6640-4bf3-b872-bf58187b91fb admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:51.362740 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1260f24-6640-4bf3-b872-bf58187b91fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:51.376742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1260f24-6640-4bf3-b872-bf58187b91fb admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:51.477268 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 897/1794] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:51 2022] POST /identity/v3/projects => generated 393 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:51.478319 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da071262-e955-4750-b483-8720e95cdd25 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:51.494356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da071262-e955-4750-b483-8720e95cdd25 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:51.506821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1260f24-6640-4bf3-b872-bf58187b91fb admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:51.508771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d1260f24-6640-4bf3-b872-bf58187b91fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:51.741506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6f2cb-ac68-4447-a899-496f39846831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:51.742183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6f2cb-ac68-4447-a899-496f39846831 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:51.742467 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6f2cb-ac68-4447-a899-496f39846831 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:51.742745 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6f2cb-ac68-4447-a899-496f39846831 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:51.743411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b6f2cb-ac68-4447-a899-496f39846831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:51.749962 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b6f2cb-ac68-4447-a899-496f39846831 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:51.772544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1260f24-6640-4bf3-b872-bf58187b91fb admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:51.814641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:51.815005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:51.815674 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:51.815957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:51.816232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] PATH_INFO: `/v3/projects/2a7acd2e8ea543cba3094bf34c4d5af5` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:51.823695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a7acd2e8ea543cba3094bf34c4d5af5)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:51.825708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:51.871360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 898/1795] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:51 2022] POST /identity/v3/users => generated 297 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:52:51.877703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6f2cb-ac68-4447-a899-496f39846831 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:51.910771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6f2cb-ac68-4447-a899-496f39846831 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:51.934441 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ffa47-a4b4-4570-a160-b20c1163869b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:51.935095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ffa47-a4b4-4570-a160-b20c1163869b admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:51.935379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ffa47-a4b4-4570-a160-b20c1163869b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:51.935732 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ffa47-a4b4-4570-a160-b20c1163869b admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:51.936331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ffa47-a4b4-4570-a160-b20c1163869b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:51.947836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ffa47-a4b4-4570-a160-b20c1163869b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:51.973895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 899/1796] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:52:51.982606 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ffa47-a4b4-4570-a160-b20c1163869b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:52.002335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:52.005949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a7acd2e8ea543cba3094bf34c4d5af5', 'request_id': 'req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:52.014174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ffa47-a4b4-4570-a160-b20c1163869b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:52.035885 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a7acd2e8ea543cba3094bf34c4d5af5', 'request_id': 'req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:52.053078 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a7acd2e8ea543cba3094bf34c4d5af5', 'request_id': 'req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:52.092480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] The token cache is being invalidate because project 2a7acd2e8ea543cba3094bf34c4d5af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:52.092884 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:52.098228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:52.106960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 898/1797] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:51 2022] DELETE /identity/v3/projects/2a7acd2e8ea543cba3094bf34c4d5af5 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:52:52.106960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:52.111780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:52.235394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c2ab17-fca6-4fe3-81c9-a5670c487d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:52.236021 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2ab17-fca6-4fe3-81c9-a5670c487d83 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:52.236306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2ab17-fca6-4fe3-81c9-a5670c487d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:52.236583 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c2ab17-fca6-4fe3-81c9-a5670c487d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:52.237233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2ab17-fca6-4fe3-81c9-a5670c487d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:52.247108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c2ab17-fca6-4fe3-81c9-a5670c487d83 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:52.276597 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 900/1798] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:52.286036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d776700-6cee-4642-8cd2-2b6a0c8ddc2b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:52.286505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d776700-6cee-4642-8cd2-2b6a0c8ddc2b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:52.286796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d776700-6cee-4642-8cd2-2b6a0c8ddc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:52.326830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c42c8-5b7c-47b7-b3f6-e35808c2f30f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:52.337289 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9d776700-6cee-4642-8cd2-2b6a0c8ddc2b None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:52.340477 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:52.341145 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:52.341794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:52.341794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b admin admin] PATH_INFO: `/v3/projects/3634641d25f749c5aeb08602d75a9ed3` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:52.343996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3634641d25f749c5aeb08602d75a9ed3)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:52.345959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:52.347472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9d776700-6cee-4642-8cd2-2b6a0c8ddc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:52.372043 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 901/1799] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:52.372043 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d776700-6cee-4642-8cd2-2b6a0c8ddc2b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:52.372043 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9d776700-6cee-4642-8cd2-2b6a0c8ddc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:52.411259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d776700-6cee-4642-8cd2-2b6a0c8ddc2b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:52.420714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862980fb-0732-4f07-92e8-7e66a83619cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:52.422117 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862980fb-0732-4f07-92e8-7e66a83619cb admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:52.422402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862980fb-0732-4f07-92e8-7e66a83619cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:52.422679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862980fb-0732-4f07-92e8-7e66a83619cb admin admin] PATH_INFO: `/v3/projects/273c312832c945fa9ad844f195881598/users/9b5b53a63e2c4c95aee759cc4ea14fa7/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:52.459556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3634641d25f749c5aeb08602d75a9ed3', 'request_id': 'req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:52.461891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862980fb-0732-4f07-92e8-7e66a83619cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=273c312832c945fa9ad844f195881598, user_id=9b5b53a63e2c4c95aee759cc4ea14fa7, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:52.464639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862980fb-0732-4f07-92e8-7e66a83619cb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:52.476614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3634641d25f749c5aeb08602d75a9ed3', 'request_id': 'req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:52.490957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3634641d25f749c5aeb08602d75a9ed3', 'request_id': 'req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:52.515054 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 902/1800] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:52 2022] PUT /identity/v3/projects/273c312832c945fa9ad844f195881598/users/9b5b53a63e2c4c95aee759cc4ea14fa7/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:52.516540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf455f-364e-4874-a3d6-b7366efb74c9 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:52.516849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf455f-364e-4874-a3d6-b7366efb74c9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:52.517128 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf455f-364e-4874-a3d6-b7366efb74c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:52.551161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b admin admin] The token cache is being invalidate because project 3634641d25f749c5aeb08602d75a9ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:52.552808 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:52.558139 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-eb3bc952-65b6-4afb-8d45-cb74bb03e12b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:52.561800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 899/1801] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:52 2022] DELETE /identity/v3/projects/3634641d25f749c5aeb08602d75a9ed3 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:52.564944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f518e-8112-4104-bddd-5feb857511e6 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:52.565444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f518e-8112-4104-bddd-5feb857511e6 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:52.565942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f518e-8112-4104-bddd-5feb857511e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:52.581243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-3faf455f-364e-4874-a3d6-b7366efb74c9 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:52.583747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3faf455f-364e-4874-a3d6-b7366efb74c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:52.648592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-277f518e-8112-4104-bddd-5feb857511e6 None None] MFA Rules not processed for user `9b5b53a63e2c4c95aee759cc4ea14fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:52.651468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-277f518e-8112-4104-bddd-5feb857511e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:52.742036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 903/1802] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:52.743249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf455f-364e-4874-a3d6-b7366efb74c9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:52.746106 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3faf455f-364e-4874-a3d6-b7366efb74c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:52.769013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf455f-364e-4874-a3d6-b7366efb74c9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:52.782537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:52.783054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:52.783410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:52.784131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:52.784131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] PATH_INFO: `/v3/users/77c4a80c845140d497c69369abd0bef5` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:52.789549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=77c4a80c845140d497c69369abd0bef5)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:52.792072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:52.807760 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 900/1803] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:52.809478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f518e-8112-4104-bddd-5feb857511e6 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:52.859240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f518e-8112-4104-bddd-5feb857511e6 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:52.877587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed6d95-d075-4594-929d-afb877428fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:52.878260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed6d95-d075-4594-929d-afb877428fce admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:52.878547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed6d95-d075-4594-929d-afb877428fce admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:52.878822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed6d95-d075-4594-929d-afb877428fce admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:52.879495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed6d95-d075-4594-929d-afb877428fce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:52.879802 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed6d95-d075-4594-929d-afb877428fce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:52.892962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed6d95-d075-4594-929d-afb877428fce admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:52.936335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 901/1804] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:52.938874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed6d95-d075-4594-929d-afb877428fce admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:52.942455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-66ed6d95-d075-4594-929d-afb877428fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:52.962387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed6d95-d075-4594-929d-afb877428fce admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:52.974337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea50d55-a6dc-46ba-a645-1abcb4b50936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:52.975704 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea50d55-a6dc-46ba-a645-1abcb4b50936 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:52.977315 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea50d55-a6dc-46ba-a645-1abcb4b50936 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:52.977315 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea50d55-a6dc-46ba-a645-1abcb4b50936 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:52.977773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea50d55-a6dc-46ba-a645-1abcb4b50936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:52.982370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea50d55-a6dc-46ba-a645-1abcb4b50936 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:53.016969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:INSERT=3,keystone:SELECT=91,keystone:DELETE=6 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:53.047296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77c4a80c845140d497c69369abd0bef5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:53.071237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77c4a80c845140d497c69369abd0bef5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:53.072165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77c4a80c845140d497c69369abd0bef5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:53.073001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77c4a80c845140d497c69369abd0bef5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:53.091269 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77c4a80c845140d497c69369abd0bef5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:53.093435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 902/1805] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:52 2022] POST /identity/v3/projects => generated 391 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:53.094870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea50d55-a6dc-46ba-a645-1abcb4b50936 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:53.096939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-9ea50d55-a6dc-46ba-a645-1abcb4b50936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:53.143158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 904/1806] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:52:52 2022] DELETE /identity/v3/users/77c4a80c845140d497c69369abd0bef5 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:53.143158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:53.144240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:53.307631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea50d55-a6dc-46ba-a645-1abcb4b50936 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:53.319541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc3127-3fea-4e2a-8260-eb662f625756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:53.319541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc3127-3fea-4e2a-8260-eb662f625756 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:53.319541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc3127-3fea-4e2a-8260-eb662f625756 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:53.319541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc3127-3fea-4e2a-8260-eb662f625756 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:53.320015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc3127-3fea-4e2a-8260-eb662f625756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:53.320107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2289e64-127f-414b-8299-a6609f062d5c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:53.320588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc3127-3fea-4e2a-8260-eb662f625756 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:53.337522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770fc01-0310-4e8c-9b54-162d934c5c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:53.338310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770fc01-0310-4e8c-9b54-162d934c5c82 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:53.338594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770fc01-0310-4e8c-9b54-162d934c5c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:53.338867 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770fc01-0310-4e8c-9b54-162d934c5c82 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:53.339522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770fc01-0310-4e8c-9b54-162d934c5c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:53.352853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770fc01-0310-4e8c-9b54-162d934c5c82 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:53.361862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 903/1807] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:53 2022] POST /identity/v3/projects => generated 383 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:53.363629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc3127-3fea-4e2a-8260-eb662f625756 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:53.419336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 905/1808] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:53 2022] POST /identity/v3/users => generated 302 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:53.424305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770fc01-0310-4e8c-9b54-162d934c5c82 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:53.478449 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770fc01-0310-4e8c-9b54-162d934c5c82 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:53.541609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc3127-3fea-4e2a-8260-eb662f625756 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:53.556398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc9908-4fc7-4965-8d0e-e1132b5555e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:53.557767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc9908-4fc7-4965-8d0e-e1132b5555e9 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:53.558226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc9908-4fc7-4965-8d0e-e1132b5555e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:53.558662 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc9908-4fc7-4965-8d0e-e1132b5555e9 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:53.559462 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc9908-4fc7-4965-8d0e-e1132b5555e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:53.566022 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc9908-4fc7-4965-8d0e-e1132b5555e9 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:53.644569 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 904/1809] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:53 2022] POST /identity/v3/users => generated 292 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:53.648298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc9908-4fc7-4965-8d0e-e1132b5555e9 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:53.673233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc9908-4fc7-4965-8d0e-e1132b5555e9 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:53.694548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef59975-1a7d-49f3-a9b0-93f09882234c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:53.695188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef59975-1a7d-49f3-a9b0-93f09882234c admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:53.695469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef59975-1a7d-49f3-a9b0-93f09882234c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:53.695744 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef59975-1a7d-49f3-a9b0-93f09882234c admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:53.696404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef59975-1a7d-49f3-a9b0-93f09882234c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:53.711794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef59975-1a7d-49f3-a9b0-93f09882234c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:53.767365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 905/1810] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:53.781996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef59975-1a7d-49f3-a9b0-93f09882234c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:53.848113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef59975-1a7d-49f3-a9b0-93f09882234c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:53.869043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769062df-b520-482a-a3ad-7bd45b975b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:53.869627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe804f8-54fc-4c70-924a-a52cb5d5e73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:53.873042 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769062df-b520-482a-a3ad-7bd45b975b0b admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:53.873419 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe804f8-54fc-4c70-924a-a52cb5d5e73f admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:53.873833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe804f8-54fc-4c70-924a-a52cb5d5e73f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:53.874267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769062df-b520-482a-a3ad-7bd45b975b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:53.874648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe804f8-54fc-4c70-924a-a52cb5d5e73f admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:53.876056 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769062df-b520-482a-a3ad-7bd45b975b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:53.877492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe804f8-54fc-4c70-924a-a52cb5d5e73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:53.881681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769062df-b520-482a-a3ad-7bd45b975b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:53.883646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe804f8-54fc-4c70-924a-a52cb5d5e73f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:53.887205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769062df-b520-482a-a3ad-7bd45b975b0b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:53.934960 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 906/1811] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 510 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:52:53.935900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 906/1812] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:53.936177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe804f8-54fc-4c70-924a-a52cb5d5e73f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:53.942666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769062df-b520-482a-a3ad-7bd45b975b0b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:53.975900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769062df-b520-482a-a3ad-7bd45b975b0b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:53.979041 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe804f8-54fc-4c70-924a-a52cb5d5e73f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:53.993053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de8371-022b-4d38-9a54-c95b0ccae4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:53.999509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de8371-022b-4d38-9a54-c95b0ccae4dd admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:54.001098 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de8371-022b-4d38-9a54-c95b0ccae4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:54.001966 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1b2674-3356-49e9-9357-dcba67d5a18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:54.002362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de8371-022b-4d38-9a54-c95b0ccae4dd admin admin] PATH_INFO: `/v3/projects/a0860d4609c642cdae4948f2ea11b5be/users/e0aa3aebc8964f0a8fa9f76210ba5507/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:54.004326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b2674-3356-49e9-9357-dcba67d5a18a admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:54.006004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b2674-3356-49e9-9357-dcba67d5a18a admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:54.006357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b2674-3356-49e9-9357-dcba67d5a18a admin admin] PATH_INFO: `/v3/projects/8052ba73815e46a9ac70e7004674efd0/users/b8935baf74ee40468cbc839ae6e8ed8e/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:54.068304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:54.079390 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b2674-3356-49e9-9357-dcba67d5a18a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8052ba73815e46a9ac70e7004674efd0, user_id=b8935baf74ee40468cbc839ae6e8ed8e, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:54.080650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de8371-022b-4d38-9a54-c95b0ccae4dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0860d4609c642cdae4948f2ea11b5be, user_id=e0aa3aebc8964f0a8fa9f76210ba5507, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:54.083692 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1b2674-3356-49e9-9357-dcba67d5a18a admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:54.106036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de8371-022b-4d38-9a54-c95b0ccae4dd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:54.117768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 907/1813] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:53 2022] PUT /identity/v3/projects/8052ba73815e46a9ac70e7004674efd0/users/b8935baf74ee40468cbc839ae6e8ed8e/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:54.131281 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f400-233d-4ae9-81f1-82638bfa62a4 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:54.134976 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f400-233d-4ae9-81f1-82638bfa62a4 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:54.138312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f400-233d-4ae9-81f1-82638bfa62a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:54.154579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 907/1814] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:53 2022] PUT /identity/v3/projects/a0860d4609c642cdae4948f2ea11b5be/users/e0aa3aebc8964f0a8fa9f76210ba5507/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:54.159170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de8371-022b-4d38-9a54-c95b0ccae4dd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:54.196892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1588f400-233d-4ae9-81f1-82638bfa62a4 None None] MFA Rules not processed for user `b8935baf74ee40468cbc839ae6e8ed8e`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:54.205211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1588f400-233d-4ae9-81f1-82638bfa62a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:54.214461 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:54.361750 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de8371-022b-4d38-9a54-c95b0ccae4dd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:54.373453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e5ac8-3ac7-43fd-8ce8-a7605caa0edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:54.395243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e5ac8-3ac7-43fd-8ce8-a7605caa0edf admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:54.397955 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e5ac8-3ac7-43fd-8ce8-a7605caa0edf admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:54.400681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3e5ac8-3ac7-43fd-8ce8-a7605caa0edf admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:54.411468 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e5ac8-3ac7-43fd-8ce8-a7605caa0edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:54.417093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3e5ac8-3ac7-43fd-8ce8-a7605caa0edf admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:54.453712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 908/1815] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:54.459130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e5ac8-3ac7-43fd-8ce8-a7605caa0edf admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:54.495585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 908/1816] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:54.496577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f400-233d-4ae9-81f1-82638bfa62a4 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:54.547626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f400-233d-4ae9-81f1-82638bfa62a4 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:54.549563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3e5ac8-3ac7-43fd-8ce8-a7605caa0edf admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:54.576977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4a491-1ed1-40ad-825f-6779719fc5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:54.584877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4a491-1ed1-40ad-825f-6779719fc5bd admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:54.585029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72646c4a-7656-468e-b61d-846f95ed1515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:54.586541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72646c4a-7656-468e-b61d-846f95ed1515 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:54.586643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4a491-1ed1-40ad-825f-6779719fc5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:54.586724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72646c4a-7656-468e-b61d-846f95ed1515 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:54.586971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72646c4a-7656-468e-b61d-846f95ed1515 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:54.588322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4a491-1ed1-40ad-825f-6779719fc5bd admin admin] PATH_INFO: `/v3/projects/a0860d4609c642cdae4948f2ea11b5be/users/e0aa3aebc8964f0a8fa9f76210ba5507/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:54.596536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72646c4a-7656-468e-b61d-846f95ed1515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:54.598627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72646c4a-7656-468e-b61d-846f95ed1515 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:54.601679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4a491-1ed1-40ad-825f-6779719fc5bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0860d4609c642cdae4948f2ea11b5be, user_id=e0aa3aebc8964f0a8fa9f76210ba5507, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:54.617970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4a491-1ed1-40ad-825f-6779719fc5bd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:54.675942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 909/1817] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:54 2022] PUT /identity/v3/projects/a0860d4609c642cdae4948f2ea11b5be/users/e0aa3aebc8964f0a8fa9f76210ba5507/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:54.689191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417b51c-a224-478e-a2c8-2b62ef6b4f4d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:54.689616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417b51c-a224-478e-a2c8-2b62ef6b4f4d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:54.690223 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417b51c-a224-478e-a2c8-2b62ef6b4f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:54.750239 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 909/1818] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:54 2022] POST /identity/v3/projects => generated 381 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:54.755195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72646c4a-7656-468e-b61d-846f95ed1515 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:54.851900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4417b51c-a224-478e-a2c8-2b62ef6b4f4d None None] MFA Rules not processed for user `e0aa3aebc8964f0a8fa9f76210ba5507`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:54.859658 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4417b51c-a224-478e-a2c8-2b62ef6b4f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:55.140938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72646c4a-7656-468e-b61d-846f95ed1515 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:55.183606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b74b6-ef29-4a8e-b9b5-84e1983062cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:55.190666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b74b6-ef29-4a8e-b9b5-84e1983062cd admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:55.192692 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b74b6-ef29-4a8e-b9b5-84e1983062cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:55.194224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b74b6-ef29-4a8e-b9b5-84e1983062cd admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:55.198134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b74b6-ef29-4a8e-b9b5-84e1983062cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:55.210966 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b74b6-ef29-4a8e-b9b5-84e1983062cd admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:55.256918 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 910/1819] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:54 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 566 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:55.265773 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417b51c-a224-478e-a2c8-2b62ef6b4f4d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:55.269653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4417b51c-a224-478e-a2c8-2b62ef6b4f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:55.309433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 910/1820] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:54 2022] POST /identity/v3/users => generated 297 bytes in 555 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:55.321288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b74b6-ef29-4a8e-b9b5-84e1983062cd admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:55.373647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b74b6-ef29-4a8e-b9b5-84e1983062cd admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:55.398380 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35369dd3-24e3-42a9-bb71-5fac8b2b65e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:55.403426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35369dd3-24e3-42a9-bb71-5fac8b2b65e6 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:55.403792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35369dd3-24e3-42a9-bb71-5fac8b2b65e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:55.404168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35369dd3-24e3-42a9-bb71-5fac8b2b65e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:55.404949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35369dd3-24e3-42a9-bb71-5fac8b2b65e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:55.416535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35369dd3-24e3-42a9-bb71-5fac8b2b65e6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:55.456558 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 911/1821] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:55.458676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417b51c-a224-478e-a2c8-2b62ef6b4f4d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:55.465165 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35369dd3-24e3-42a9-bb71-5fac8b2b65e6 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:55.471598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b2e60-3aab-4d56-a23a-13a879269860 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:55.472356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b2e60-3aab-4d56-a23a-13a879269860 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:55.472639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b2e60-3aab-4d56-a23a-13a879269860 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:55.472937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b2e60-3aab-4d56-a23a-13a879269860 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:55.508260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b2e60-3aab-4d56-a23a-13a879269860 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:55.516906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b2e60-3aab-4d56-a23a-13a879269860 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:55.517541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35369dd3-24e3-42a9-bb71-5fac8b2b65e6 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:55.545297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506464ab-4e18-4fc5-a33e-4b0b4f1be042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:55.551876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506464ab-4e18-4fc5-a33e-4b0b4f1be042 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:55.552064 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506464ab-4e18-4fc5-a33e-4b0b4f1be042 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:55.552255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506464ab-4e18-4fc5-a33e-4b0b4f1be042 admin admin] PATH_INFO: `/v3/projects/5e141024dc66465cb9fc65ce702525d8/users/4b25ff910acc407481571049b4e661f3/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:55.557379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 911/1822] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:55 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:52:55.632648 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506464ab-4e18-4fc5-a33e-4b0b4f1be042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e141024dc66465cb9fc65ce702525d8, user_id=4b25ff910acc407481571049b4e661f3, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:55.644185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506464ab-4e18-4fc5-a33e-4b0b4f1be042 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:55.689316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b2e60-3aab-4d56-a23a-13a879269860 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:55.691410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-970b2e60-3aab-4d56-a23a-13a879269860 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:55.691965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 912/1823] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:55 2022] PUT /identity/v3/projects/5e141024dc66465cb9fc65ce702525d8/users/4b25ff910acc407481571049b4e661f3/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:52:55.696401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506464ab-4e18-4fc5-a33e-4b0b4f1be042 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:55.730518 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:55.751402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b2e60-3aab-4d56-a23a-13a879269860 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:55.836810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:55.893716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0f611-784e-459d-a0f7-aa34f3050ec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:55.894785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0f611-784e-459d-a0f7-aa34f3050ec6 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:55.895335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0f611-784e-459d-a0f7-aa34f3050ec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:55.895827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0f611-784e-459d-a0f7-aa34f3050ec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:55.899953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c3f0f611-784e-459d-a0f7-aa34f3050ec6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:55.908820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506464ab-4e18-4fc5-a33e-4b0b4f1be042 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:55.933683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec9729-80a4-4459-b6af-1afc6bf1af7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:55.938255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec9729-80a4-4459-b6af-1afc6bf1af7a admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:55.939731 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec9729-80a4-4459-b6af-1afc6bf1af7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:55.940012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edec9729-80a4-4459-b6af-1afc6bf1af7a admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:55.942940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec9729-80a4-4459-b6af-1afc6bf1af7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:55.947970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edec9729-80a4-4459-b6af-1afc6bf1af7a admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:55.979709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 913/1824] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:55 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:55.982413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec9729-80a4-4459-b6af-1afc6bf1af7a admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:55.989422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-edec9729-80a4-4459-b6af-1afc6bf1af7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:56.019829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edec9729-80a4-4459-b6af-1afc6bf1af7a admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:56.032533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f643d-ba0b-4a75-a25e-01e8017c1435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:56.033183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f643d-ba0b-4a75-a25e-01e8017c1435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:56.033472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f643d-ba0b-4a75-a25e-01e8017c1435 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:56.033792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f643d-ba0b-4a75-a25e-01e8017c1435 admin admin] PATH_INFO: `/v3/projects/6dee55a1c3454a8baf3346e688416405` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:56.036741 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f643d-ba0b-4a75-a25e-01e8017c1435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dee55a1c3454a8baf3346e688416405)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:56.038703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f643d-ba0b-4a75-a25e-01e8017c1435 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:56.069989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0f611-784e-459d-a0f7-aa34f3050ec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:56.072457 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0f611-784e-459d-a0f7-aa34f3050ec6 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:56.109244 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 912/1825] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:55 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:52:56.112830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0f611-784e-459d-a0f7-aa34f3050ec6 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:56.149016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0f611-784e-459d-a0f7-aa34f3050ec6 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:56.163454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d743ec8-6c10-47b8-bbbf-e09f531b7476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:56.164495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d743ec8-6c10-47b8-bbbf-e09f531b7476 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:56.178689 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d743ec8-6c10-47b8-bbbf-e09f531b7476 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:56.179185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d743ec8-6c10-47b8-bbbf-e09f531b7476 admin admin] PATH_INFO: `/v3/projects/5e141024dc66465cb9fc65ce702525d8/users/4b25ff910acc407481571049b4e661f3/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:56.186357 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d743ec8-6c10-47b8-bbbf-e09f531b7476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e141024dc66465cb9fc65ce702525d8, user_id=4b25ff910acc407481571049b4e661f3, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:56.202354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d743ec8-6c10-47b8-bbbf-e09f531b7476 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:56.221416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d43f643d-ba0b-4a75-a25e-01e8017c1435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dee55a1c3454a8baf3346e688416405', 'request_id': 'req-d43f643d-ba0b-4a75-a25e-01e8017c1435'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:56.237251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 913/1826] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:56 2022] PUT /identity/v3/projects/5e141024dc66465cb9fc65ce702525d8/users/4b25ff910acc407481571049b4e661f3/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:56.248540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d43f643d-ba0b-4a75-a25e-01e8017c1435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dee55a1c3454a8baf3346e688416405', 'request_id': 'req-d43f643d-ba0b-4a75-a25e-01e8017c1435'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:56.254727 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfbb08-800a-4fa0-b18c-d44998473bd6 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:56.255032 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfbb08-800a-4fa0-b18c-d44998473bd6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:56.255332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfbb08-800a-4fa0-b18c-d44998473bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:56.266257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d43f643d-ba0b-4a75-a25e-01e8017c1435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dee55a1c3454a8baf3346e688416405', 'request_id': 'req-d43f643d-ba0b-4a75-a25e-01e8017c1435'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:56.292118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d43f643d-ba0b-4a75-a25e-01e8017c1435 admin admin] The token cache is being invalidate because project 6dee55a1c3454a8baf3346e688416405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:52:56.296151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d43f643d-ba0b-4a75-a25e-01e8017c1435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:56.296151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d43f643d-ba0b-4a75-a25e-01e8017c1435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:52:56.296151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 914/1827] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:52:55 2022] DELETE /identity/v3/projects/6dee55a1c3454a8baf3346e688416405 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:56.313403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276ac2a-e0d4-42ea-b977-08b77ea1ffe2 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:56.313715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276ac2a-e0d4-42ea-b977-08b77ea1ffe2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:56.314008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276ac2a-e0d4-42ea-b977-08b77ea1ffe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:56.353352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-c276ac2a-e0d4-42ea-b977-08b77ea1ffe2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:56.358661 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c276ac2a-e0d4-42ea-b977-08b77ea1ffe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:56.392911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7cbfbb08-800a-4fa0-b18c-d44998473bd6 None None] MFA Rules not processed for user `4b25ff910acc407481571049b4e661f3`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:56.394872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7cbfbb08-800a-4fa0-b18c-d44998473bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:56.631261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 915/1828] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:56.632766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1f90e-464f-4457-9272-85dcd8b25655 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:56.633656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1f90e-464f-4457-9272-85dcd8b25655 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:56.633656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1f90e-464f-4457-9272-85dcd8b25655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:56.657644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 914/1829] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:56.674542 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfbb08-800a-4fa0-b18c-d44998473bd6 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:56.680037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7cbfbb08-800a-4fa0-b18c-d44998473bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:56.692036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-37d1f90e-464f-4457-9272-85dcd8b25655 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:56.700431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-37d1f90e-464f-4457-9272-85dcd8b25655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:56.761306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 916/1830] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:56.762606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1f90e-464f-4457-9272-85dcd8b25655 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:56.788470 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1f90e-464f-4457-9272-85dcd8b25655 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:56.814587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38086156-ce4f-4a87-b85f-3e5f0520e38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:56.815509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38086156-ce4f-4a87-b85f-3e5f0520e38b admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:56.816060 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38086156-ce4f-4a87-b85f-3e5f0520e38b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:56.816502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38086156-ce4f-4a87-b85f-3e5f0520e38b admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:56.817338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38086156-ce4f-4a87-b85f-3e5f0520e38b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:52:56.817828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38086156-ce4f-4a87-b85f-3e5f0520e38b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:56.819619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38086156-ce4f-4a87-b85f-3e5f0520e38b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:56.864911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 917/1831] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:52:56.870840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38086156-ce4f-4a87-b85f-3e5f0520e38b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:56.892173 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38086156-ce4f-4a87-b85f-3e5f0520e38b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:56.934261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712c0c0-ad32-4235-adf8-82b9f6cecee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:56.935044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712c0c0-ad32-4235-adf8-82b9f6cecee9 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:56.935310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712c0c0-ad32-4235-adf8-82b9f6cecee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:56.935579 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712c0c0-ad32-4235-adf8-82b9f6cecee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:56.936209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712c0c0-ad32-4235-adf8-82b9f6cecee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:56.939516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712c0c0-ad32-4235-adf8-82b9f6cecee9 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:56.941803 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfbb08-800a-4fa0-b18c-d44998473bd6 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:56.956587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0f255-bd38-46f9-beed-ba9df1d04f83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:56.963926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0f255-bd38-46f9-beed-ba9df1d04f83 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:56.965649 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0f255-bd38-46f9-beed-ba9df1d04f83 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:56.971061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0f255-bd38-46f9-beed-ba9df1d04f83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:57.029680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:57.041056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d0f255-bd38-46f9-beed-ba9df1d04f83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:57.044835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d0f255-bd38-46f9-beed-ba9df1d04f83 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:57.097820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:57.115931 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 918/1832] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:56 2022] POST /identity/v3/projects => generated 381 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:57.118695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712c0c0-ad32-4235-adf8-82b9f6cecee9 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:57.123302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 915/1833] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:57.219923 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfb106-43b3-4a9e-b866-e3e6a911f0bf None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:57.220235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfb106-43b3-4a9e-b866-e3e6a911f0bf None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:57.220508 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfb106-43b3-4a9e-b866-e3e6a911f0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:57.324023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-21dfb106-43b3-4a9e-b866-e3e6a911f0bf None None] MFA Rules not processed for user `531ebc79bf424e9b8762b2c7ec7fe8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:57.326507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-21dfb106-43b3-4a9e-b866-e3e6a911f0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:57.365359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712c0c0-ad32-4235-adf8-82b9f6cecee9 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:57.386116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e3e71-59dc-478c-830e-e774bda1ad8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:57.386754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e3e71-59dc-478c-830e-e774bda1ad8b admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:57.387042 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e3e71-59dc-478c-830e-e774bda1ad8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:57.387294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e3e71-59dc-478c-830e-e774bda1ad8b admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:57.387957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e3e71-59dc-478c-830e-e774bda1ad8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:57.390639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e3e71-59dc-478c-830e-e774bda1ad8b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:57.545321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 919/1834] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:57 2022] POST /identity/v3/users => generated 291 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:57.555974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e3e71-59dc-478c-830e-e774bda1ad8b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:57.566596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b75e3e71-59dc-478c-830e-e774bda1ad8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:57.765755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 916/1835] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Sep 8 13:52:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 548 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:52:57.768951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfb106-43b3-4a9e-b866-e3e6a911f0bf None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:57.804407 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfb106-43b3-4a9e-b866-e3e6a911f0bf None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:57.829861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c54819-18a6-4061-845d-135f86e17202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:57.830716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c54819-18a6-4061-845d-135f86e17202 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:57.834423 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c54819-18a6-4061-845d-135f86e17202 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:57.836193 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c54819-18a6-4061-845d-135f86e17202 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:57.839968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c54819-18a6-4061-845d-135f86e17202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:57.841640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c54819-18a6-4061-845d-135f86e17202 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:57.907800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 917/1836] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:57.913882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c54819-18a6-4061-845d-135f86e17202 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:57.947296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e3e71-59dc-478c-830e-e774bda1ad8b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:57.957527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c54819-18a6-4061-845d-135f86e17202 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:57.968529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0ac98-d73b-4ad6-8a18-e79ffc3b2210 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:57.969179 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0ac98-d73b-4ad6-8a18-e79ffc3b2210 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:57.969456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0ac98-d73b-4ad6-8a18-e79ffc3b2210 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:57.969742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0ac98-d73b-4ad6-8a18-e79ffc3b2210 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:57.973628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042dc2a-f250-4dd6-9f2b-a337b08ac4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:57.973628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042dc2a-f250-4dd6-9f2b-a337b08ac4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:57.973845 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042dc2a-f250-4dd6-9f2b-a337b08ac4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:57.974345 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042dc2a-f250-4dd6-9f2b-a337b08ac4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:57.976661 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042dc2a-f250-4dd6-9f2b-a337b08ac4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:57.979338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-bde0ac98-d73b-4ad6-8a18-e79ffc3b2210 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:57.980698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042dc2a-f250-4dd6-9f2b-a337b08ac4e9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:58.024627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 918/1837] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:58.026007 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042dc2a-f250-4dd6-9f2b-a337b08ac4e9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:58.043046 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0042dc2a-f250-4dd6-9f2b-a337b08ac4e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:58.081487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042dc2a-f250-4dd6-9f2b-a337b08ac4e9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:58.096622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd37aa-8cbb-4512-b4d7-f9fa07e74925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:58.097492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd37aa-8cbb-4512-b4d7-f9fa07e74925 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:58.098057 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd37aa-8cbb-4512-b4d7-f9fa07e74925 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:58.099355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd37aa-8cbb-4512-b4d7-f9fa07e74925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:58.111017 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd37aa-8cbb-4512-b4d7-f9fa07e74925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:58.112764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd37aa-8cbb-4512-b4d7-f9fa07e74925 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:58.124006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 919/1838] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:58.124998 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd37aa-8cbb-4512-b4d7-f9fa07e74925 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:58.148510 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd37aa-8cbb-4512-b4d7-f9fa07e74925 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:58.175147 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620968f2-97ff-42db-8aac-a78caf4e6fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:58.175969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620968f2-97ff-42db-8aac-a78caf4e6fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:58.204289 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620968f2-97ff-42db-8aac-a78caf4e6fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:58.207965 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620968f2-97ff-42db-8aac-a78caf4e6fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:58.238766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620968f2-97ff-42db-8aac-a78caf4e6fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:58.244619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620968f2-97ff-42db-8aac-a78caf4e6fb0 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:58.265096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0ac98-d73b-4ad6-8a18-e79ffc3b2210 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:58.273256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 920/1839] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:58.273884 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0ac98-d73b-4ad6-8a18-e79ffc3b2210 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:58.281601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620968f2-97ff-42db-8aac-a78caf4e6fb0 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:58.324921 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 920/1840] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 779 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:58.326102 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620968f2-97ff-42db-8aac-a78caf4e6fb0 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:58.350025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fae8ac-3112-47f5-90aa-c30d6e723b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:58.350858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fae8ac-3112-47f5-90aa-c30d6e723b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:58.351245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fae8ac-3112-47f5-90aa-c30d6e723b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:58.351637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fae8ac-3112-47f5-90aa-c30d6e723b75 admin admin] PATH_INFO: `/v3/projects/8ba6ca7d485c404dbe3ed5873c2142f6/users/cde78890e14b4803a7ea52cddce250de/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:58.406504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fae8ac-3112-47f5-90aa-c30d6e723b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba6ca7d485c404dbe3ed5873c2142f6, user_id=cde78890e14b4803a7ea52cddce250de, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:58.414812 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fae8ac-3112-47f5-90aa-c30d6e723b75 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:58.465200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 921/1841] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:58 2022] PUT /identity/v3/projects/8ba6ca7d485c404dbe3ed5873c2142f6/users/cde78890e14b4803a7ea52cddce250de/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:58.471366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171583aa-668f-4559-9cb2-bdef81e2b205 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:58.471863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171583aa-668f-4559-9cb2-bdef81e2b205 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:58.471957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171583aa-668f-4559-9cb2-bdef81e2b205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:58.600635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-171583aa-668f-4559-9cb2-bdef81e2b205 None None] MFA Rules not processed for user `cde78890e14b4803a7ea52cddce250de`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:58.606697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-171583aa-668f-4559-9cb2-bdef81e2b205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:58.697029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:58.818319 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 921/1842] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:58.822671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fae8ac-3112-47f5-90aa-c30d6e723b75 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:58.985678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fae8ac-3112-47f5-90aa-c30d6e723b75 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:58.998544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40ee85-e8dc-4338-93c5-ce055400b0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:59.001833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40ee85-e8dc-4338-93c5-ce055400b0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:59.002913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40ee85-e8dc-4338-93c5-ce055400b0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:59.004339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40ee85-e8dc-4338-93c5-ce055400b0f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:59.007537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40ee85-e8dc-4338-93c5-ce055400b0f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:59.012740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40ee85-e8dc-4338-93c5-ce055400b0f4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:59.066380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 922/1843] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:52:58 2022] POST /identity/v3/projects => generated 381 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:59.070257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171583aa-668f-4559-9cb2-bdef81e2b205 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:59.090230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171583aa-668f-4559-9cb2-bdef81e2b205 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:59.245397 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7e543-7fe1-42fc-bd34-cea183ac6881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:59.246113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e543-7fe1-42fc-bd34-cea183ac6881 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:59.246400 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e543-7fe1-42fc-bd34-cea183ac6881 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:59.246677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7e543-7fe1-42fc-bd34-cea183ac6881 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:59.247343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7e543-7fe1-42fc-bd34-cea183ac6881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:59.256159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7e543-7fe1-42fc-bd34-cea183ac6881 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:59.310660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 922/1844] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:52:59 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:52:59.314377 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40ee85-e8dc-4338-93c5-ce055400b0f4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:59.334621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40ee85-e8dc-4338-93c5-ce055400b0f4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:59.343804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94235604-0e89-4a6b-963d-e876a99fba03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:59.344441 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94235604-0e89-4a6b-963d-e876a99fba03 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:59.344721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94235604-0e89-4a6b-963d-e876a99fba03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:59.344994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94235604-0e89-4a6b-963d-e876a99fba03 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:59.345834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94235604-0e89-4a6b-963d-e876a99fba03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:59.353380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94235604-0e89-4a6b-963d-e876a99fba03 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:59.398991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 923/1845] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:52:59.402932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7e543-7fe1-42fc-bd34-cea183ac6881 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:59.436343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7e543-7fe1-42fc-bd34-cea183ac6881 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:59.457084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600e242-2f57-4a5a-8dcb-60be70f91a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:59.459514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600e242-2f57-4a5a-8dcb-60be70f91a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:59.460793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600e242-2f57-4a5a-8dcb-60be70f91a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:59.462370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600e242-2f57-4a5a-8dcb-60be70f91a9e admin admin] PATH_INFO: `/v3/projects/f48ad170248c4705b811cd802616e460/users/1a0027dadb4549fe862c63ce93823e98/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:59.500314 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600e242-2f57-4a5a-8dcb-60be70f91a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48ad170248c4705b811cd802616e460, user_id=1a0027dadb4549fe862c63ce93823e98, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:59.506346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600e242-2f57-4a5a-8dcb-60be70f91a9e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:59.541630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 923/1846] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:59 2022] PUT /identity/v3/projects/f48ad170248c4705b811cd802616e460/users/1a0027dadb4549fe862c63ce93823e98/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:59.545541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94235604-0e89-4a6b-963d-e876a99fba03 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:59.562790 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94235604-0e89-4a6b-963d-e876a99fba03 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:59.584588 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:59.768495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c219c17-c62f-4e4d-a4b4-de5f94fe2fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:59.771946 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c219c17-c62f-4e4d-a4b4-de5f94fe2fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:59.772574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c219c17-c62f-4e4d-a4b4-de5f94fe2fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:59.773094 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c219c17-c62f-4e4d-a4b4-de5f94fe2fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:59.775830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c219c17-c62f-4e4d-a4b4-de5f94fe2fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:59.781128 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c219c17-c62f-4e4d-a4b4-de5f94fe2fa0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:59.809214 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 924/1847] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:52:59 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:52:59.810514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600e242-2f57-4a5a-8dcb-60be70f91a9e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:59.828568 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600e242-2f57-4a5a-8dcb-60be70f91a9e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:59.846262 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbec8c-82b5-40d5-a9a3-d592db8971c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:59.847118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbec8c-82b5-40d5-a9a3-d592db8971c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:59.847404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbec8c-82b5-40d5-a9a3-d592db8971c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:59.847674 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dbec8c-82b5-40d5-a9a3-d592db8971c4 admin admin] PATH_INFO: `/v3/projects/f48ad170248c4705b811cd802616e460/users/1a0027dadb4549fe862c63ce93823e98/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:59.852949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbec8c-82b5-40d5-a9a3-d592db8971c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48ad170248c4705b811cd802616e460, user_id=1a0027dadb4549fe862c63ce93823e98, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:59.858887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dbec8c-82b5-40d5-a9a3-d592db8971c4 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:59.895860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 924/1848] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:52:59 2022] PUT /identity/v3/projects/f48ad170248c4705b811cd802616e460/users/1a0027dadb4549fe862c63ce93823e98/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:52:59.902795 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83637d5-7043-43ad-9b67-5fbc6fa079ca None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:59.903437 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83637d5-7043-43ad-9b67-5fbc6fa079ca None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:59.903437 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83637d5-7043-43ad-9b67-5fbc6fa079ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:59.975099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e83637d5-7043-43ad-9b67-5fbc6fa079ca None None] MFA Rules not processed for user `1a0027dadb4549fe862c63ce93823e98`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:59.976512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e83637d5-7043-43ad-9b67-5fbc6fa079ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:00.164154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 925/1849] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:52:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:00.335984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbec8c-82b5-40d5-a9a3-d592db8971c4 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:00.448651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dbec8c-82b5-40d5-a9a3-d592db8971c4 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:00.457725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d58c7b-268d-41e0-922a-d23dbb26de24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:00.458384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d58c7b-268d-41e0-922a-d23dbb26de24 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:00.458683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d58c7b-268d-41e0-922a-d23dbb26de24 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:00.458930 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d58c7b-268d-41e0-922a-d23dbb26de24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:00.487174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d58c7b-268d-41e0-922a-d23dbb26de24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:00.488641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d58c7b-268d-41e0-922a-d23dbb26de24 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:00.534122 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 925/1850] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:00.935064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83637d5-7043-43ad-9b67-5fbc6fa079ca None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:00.945965 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83637d5-7043-43ad-9b67-5fbc6fa079ca None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:00.971258 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b714f-c52d-40b0-b2f0-deb609cdea71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:00.971678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b714f-c52d-40b0-b2f0-deb609cdea71 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:00.971999 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b714f-c52d-40b0-b2f0-deb609cdea71 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:00.972217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b714f-c52d-40b0-b2f0-deb609cdea71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:01.095757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b714f-c52d-40b0-b2f0-deb609cdea71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:01.097160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b714f-c52d-40b0-b2f0-deb609cdea71 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:01.127692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 926/1851] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:01.732631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3f864-a765-437d-9fa3-ca47b8149fc0 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:01.734843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3f864-a765-437d-9fa3-ca47b8149fc0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:01.735424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3f864-a765-437d-9fa3-ca47b8149fc0 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:01.737887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 926/1852] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 13:53:01 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:01.780131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c4aad-2fb0-4aa7-b2d3-5c313de22a9f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:01.782058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c4aad-2fb0-4aa7-b2d3-5c313de22a9f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:01.783275 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c4aad-2fb0-4aa7-b2d3-5c313de22a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:01.825260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a21c4aad-2fb0-4aa7-b2d3-5c313de22a9f None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:01.828847 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a21c4aad-2fb0-4aa7-b2d3-5c313de22a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:01.887018 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:02.003899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 927/1853] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 13:53:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:02.021065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3f864-a765-437d-9fa3-ca47b8149fc0 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:02.027287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c2a3f864-a765-437d-9fa3-ca47b8149fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:02.096838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3f864-a765-437d-9fa3-ca47b8149fc0 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:02.127126 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31bebd-78f4-4e9a-af86-696deba52d1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:02.128206 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31bebd-78f4-4e9a-af86-696deba52d1f service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:02.128671 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31bebd-78f4-4e9a-af86-696deba52d1f service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:02.129126 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31bebd-78f4-4e9a-af86-696deba52d1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:02.166502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31bebd-78f4-4e9a-af86-696deba52d1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:02.169251 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f31bebd-78f4-4e9a-af86-696deba52d1f service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:02.210387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 927/1854] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:03.021727 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c4aad-2fb0-4aa7-b2d3-5c313de22a9f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:03.023336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a21c4aad-2fb0-4aa7-b2d3-5c313de22a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:03.068938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31bebd-78f4-4e9a-af86-696deba52d1f service nova] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:03.070556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7f31bebd-78f4-4e9a-af86-696deba52d1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:03.078571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:03.281688 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c4aad-2fb0-4aa7-b2d3-5c313de22a9f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:03.299013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f545-9b9e-4124-8406-8b74695a42e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:03.301085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f545-9b9e-4124-8406-8b74695a42e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:03.302821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f545-9b9e-4124-8406-8b74695a42e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:03.303339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f545-9b9e-4124-8406-8b74695a42e9 admin admin] PATH_INFO: `/v3/users/9b5b53a63e2c4c95aee759cc4ea14fa7` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:03.307924 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f545-9b9e-4124-8406-8b74695a42e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b5b53a63e2c4c95aee759cc4ea14fa7)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:03.313642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f545-9b9e-4124-8406-8b74695a42e9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:03.330092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31bebd-78f4-4e9a-af86-696deba52d1f service nova] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:03.350002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e5ab91-1319-4792-b407-5ad7500377a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:03.350848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5ab91-1319-4792-b407-5ad7500377a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:03.351244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5ab91-1319-4792-b407-5ad7500377a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:03.351862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e5ab91-1319-4792-b407-5ad7500377a0 admin admin] PATH_INFO: `/v3/users/b8935baf74ee40468cbc839ae6e8ed8e` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:03.354186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5ab91-1319-4792-b407-5ad7500377a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8935baf74ee40468cbc839ae6e8ed8e)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:03.361934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e5ab91-1319-4792-b407-5ad7500377a0 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:03.454241 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f94f545-9b9e-4124-8406-8b74695a42e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b5b53a63e2c4c95aee759cc4ea14fa7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:03.475411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f94f545-9b9e-4124-8406-8b74695a42e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b5b53a63e2c4c95aee759cc4ea14fa7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:03.476053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f94f545-9b9e-4124-8406-8b74695a42e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b5b53a63e2c4c95aee759cc4ea14fa7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:03.476886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f94f545-9b9e-4124-8406-8b74695a42e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b5b53a63e2c4c95aee759cc4ea14fa7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:03.493082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f94f545-9b9e-4124-8406-8b74695a42e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b5b53a63e2c4c95aee759cc4ea14fa7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:03.503245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e1e5ab91-1319-4792-b407-5ad7500377a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8935baf74ee40468cbc839ae6e8ed8e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:03.526970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 928/1855] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:03 2022] DELETE /identity/v3/users/9b5b53a63e2c4c95aee759cc4ea14fa7 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:03.528991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3f8c0-616b-4c4c-983c-8e273fdf3865 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:03.529299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3f8c0-616b-4c4c-983c-8e273fdf3865 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:03.529658 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3f8c0-616b-4c4c-983c-8e273fdf3865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:03.531286 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e1e5ab91-1319-4792-b407-5ad7500377a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8935baf74ee40468cbc839ae6e8ed8e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:03.532194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e1e5ab91-1319-4792-b407-5ad7500377a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8935baf74ee40468cbc839ae6e8ed8e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:03.533429 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e1e5ab91-1319-4792-b407-5ad7500377a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8935baf74ee40468cbc839ae6e8ed8e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:03.545506 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e1e5ab91-1319-4792-b407-5ad7500377a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8935baf74ee40468cbc839ae6e8ed8e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:03.570973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 928/1856] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:03 2022] DELETE /identity/v3/users/b8935baf74ee40468cbc839ae6e8ed8e => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:03.571336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-bbf3f8c0-616b-4c4c-983c-8e273fdf3865 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:03.572583 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bbf3f8c0-616b-4c4c-983c-8e273fdf3865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:03.572731 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7741516-2a91-4129-b5dd-4860071bc0e0 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:03.573067 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7741516-2a91-4129-b5dd-4860071bc0e0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:03.573444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7741516-2a91-4129-b5dd-4860071bc0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:03.598137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-e7741516-2a91-4129-b5dd-4860071bc0e0 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:03.599795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e7741516-2a91-4129-b5dd-4860071bc0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:03.660913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:03.709951 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 929/1857] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:03.732129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 929/1858] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:03.762670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3f8c0-616b-4c4c-983c-8e273fdf3865 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:03.765121 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bbf3f8c0-616b-4c4c-983c-8e273fdf3865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:03.773783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7741516-2a91-4129-b5dd-4860071bc0e0 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:03.780606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e7741516-2a91-4129-b5dd-4860071bc0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:03.828595 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:03.863388 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7741516-2a91-4129-b5dd-4860071bc0e0 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:03.868787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c359f90-82f1-4a83-b615-a599f3c8931c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:03.869293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c359f90-82f1-4a83-b615-a599f3c8931c service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:03.869553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c359f90-82f1-4a83-b615-a599f3c8931c service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:03.869965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c359f90-82f1-4a83-b615-a599f3c8931c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:03.894488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c359f90-82f1-4a83-b615-a599f3c8931c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:03.904680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3f8c0-616b-4c4c-983c-8e273fdf3865 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:03.912227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c359f90-82f1-4a83-b615-a599f3c8931c service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:03.925648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08f375-a73e-4944-b8f2-3d3079c0f133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:03.926688 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08f375-a73e-4944-b8f2-3d3079c0f133 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:03.926808 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 930/1859] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:03.926898 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08f375-a73e-4944-b8f2-3d3079c0f133 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:03.927073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08f375-a73e-4944-b8f2-3d3079c0f133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:03.939012 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08f375-a73e-4944-b8f2-3d3079c0f133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:03.940463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08f375-a73e-4944-b8f2-3d3079c0f133 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:03.954148 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 930/1860] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:04.567509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2fbeb-bb1e-42cf-a256-2a3d6c6e314a None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:04.568369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2fbeb-bb1e-42cf-a256-2a3d6c6e314a None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:04.568369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2fbeb-bb1e-42cf-a256-2a3d6c6e314a None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:04.569397 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 931/1861] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 13:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:04.606432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f898c09-dbcb-40fb-aaab-dd9f8e1b3027 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:04.607011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f898c09-dbcb-40fb-aaab-dd9f8e1b3027 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:04.607011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f898c09-dbcb-40fb-aaab-dd9f8e1b3027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:04.642908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4f898c09-dbcb-40fb-aaab-dd9f8e1b3027 None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:04.645653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4f898c09-dbcb-40fb-aaab-dd9f8e1b3027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:04.817984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2fbeb-bb1e-42cf-a256-2a3d6c6e314a None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:04.822659 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-9dd2fbeb-bb1e-42cf-a256-2a3d6c6e314a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:04.920985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 931/1862] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 13:53:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:04.930610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2fbeb-bb1e-42cf-a256-2a3d6c6e314a None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:04.942935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:04.943808 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:04.944184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:04.944554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] PATH_INFO: `/v3/projects/273c312832c945fa9ad844f195881598` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:04.945851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=273c312832c945fa9ad844f195881598)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:04.948408 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:05.085452 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '273c312832c945fa9ad844f195881598', 'request_id': 'req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.093419 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '273c312832c945fa9ad844f195881598', 'request_id': 'req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.102758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '273c312832c945fa9ad844f195881598', 'request_id': 'req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.119672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] The token cache is being invalidate because project 273c312832c945fa9ad844f195881598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:05.119988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.120763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.126255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 932/1863] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:04 2022] DELETE /identity/v3/projects/273c312832c945fa9ad844f195881598 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:05.126964 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f898c09-dbcb-40fb-aaab-dd9f8e1b3027 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:05.150457 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f898c09-dbcb-40fb-aaab-dd9f8e1b3027 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:05.255931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:05.256572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:05.258091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:05.258091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] PATH_INFO: `/v3/users/e0aa3aebc8964f0a8fa9f76210ba5507` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:05.259052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0aa3aebc8964f0a8fa9f76210ba5507)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:05.263296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:05.289543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:05.301773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:05.303274 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:05.320947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663769a8-50b8-4dcb-b5f3-8b3d77ddd641 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:05.325678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:05.326722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:05.327068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:05.327844 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] PATH_INFO: `/v3/projects/8052ba73815e46a9ac70e7004674efd0` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:05.329626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8052ba73815e46a9ac70e7004674efd0)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:05.331552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:05.357714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0aa3aebc8964f0a8fa9f76210ba5507'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.368351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0aa3aebc8964f0a8fa9f76210ba5507'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.369087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0aa3aebc8964f0a8fa9f76210ba5507'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.369725 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0aa3aebc8964f0a8fa9f76210ba5507'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.385435 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0aa3aebc8964f0a8fa9f76210ba5507'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.428090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 932/1864] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:05 2022] DELETE /identity/v3/users/e0aa3aebc8964f0a8fa9f76210ba5507 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:05.428298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8052ba73815e46a9ac70e7004674efd0', 'request_id': 'req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.428406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:05.428406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:05.438934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8052ba73815e46a9ac70e7004674efd0', 'request_id': 'req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.451145 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8052ba73815e46a9ac70e7004674efd0', 'request_id': 'req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.490646 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] The token cache is being invalidate because project 8052ba73815e46a9ac70e7004674efd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:05.491089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.498233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.504143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 933/1865] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:05 2022] DELETE /identity/v3/projects/8052ba73815e46a9ac70e7004674efd0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:05.509782 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:05.511722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:05.601435 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:05.627183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980fe24-7214-40cb-82ed-99c22f764ed6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:05.645456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5237c-7a0b-4ccd-9846-54a60476c5f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:05.646123 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5237c-7a0b-4ccd-9846-54a60476c5f9 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:05.646418 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5237c-7a0b-4ccd-9846-54a60476c5f9 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:05.646986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5237c-7a0b-4ccd-9846-54a60476c5f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:05.654778 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a7e5237c-7a0b-4ccd-9846-54a60476c5f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:05.669221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb7ee9-27fa-44a4-90f8-001ae1aa1419 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:05.675105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5237c-7a0b-4ccd-9846-54a60476c5f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:05.678769 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5237c-7a0b-4ccd-9846-54a60476c5f9 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:05.704156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 933/1866] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:05.755934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e458c46-52fd-46d5-9b86-1f29be4c6864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:05.756789 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e458c46-52fd-46d5-9b86-1f29be4c6864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:05.756979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e458c46-52fd-46d5-9b86-1f29be4c6864 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:05.757517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e458c46-52fd-46d5-9b86-1f29be4c6864 admin admin] PATH_INFO: `/v3/users/4b25ff910acc407481571049b4e661f3` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:05.759043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e458c46-52fd-46d5-9b86-1f29be4c6864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b25ff910acc407481571049b4e661f3)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:05.761459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e458c46-52fd-46d5-9b86-1f29be4c6864 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:05.844200 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8e458c46-52fd-46d5-9b86-1f29be4c6864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b25ff910acc407481571049b4e661f3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.855600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8e458c46-52fd-46d5-9b86-1f29be4c6864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b25ff910acc407481571049b4e661f3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.856771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8e458c46-52fd-46d5-9b86-1f29be4c6864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b25ff910acc407481571049b4e661f3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.857947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8e458c46-52fd-46d5-9b86-1f29be4c6864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b25ff910acc407481571049b4e661f3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.869390 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8e458c46-52fd-46d5-9b86-1f29be4c6864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b25ff910acc407481571049b4e661f3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.872176 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5237c-7a0b-4ccd-9846-54a60476c5f9 service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:05.873892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a7e5237c-7a0b-4ccd-9846-54a60476c5f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:05.893249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5237c-7a0b-4ccd-9846-54a60476c5f9 service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:05.902494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 934/1867] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:05 2022] DELETE /identity/v3/users/4b25ff910acc407481571049b4e661f3 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:05.911314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd5e42-1f06-455c-bbb1-d0e99c489c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:05.912026 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd5e42-1f06-455c-bbb1-d0e99c489c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:05.912974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd5e42-1f06-455c-bbb1-d0e99c489c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:05.912974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd5e42-1f06-455c-bbb1-d0e99c489c45 admin admin] PATH_INFO: `/v3/projects/a0860d4609c642cdae4948f2ea11b5be` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:05.914348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd5e42-1f06-455c-bbb1-d0e99c489c45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0860d4609c642cdae4948f2ea11b5be)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:05.916304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd5e42-1f06-455c-bbb1-d0e99c489c45 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:05.978621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-febd5e42-1f06-455c-bbb1-d0e99c489c45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0860d4609c642cdae4948f2ea11b5be', 'request_id': 'req-febd5e42-1f06-455c-bbb1-d0e99c489c45'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:05.990836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-febd5e42-1f06-455c-bbb1-d0e99c489c45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0860d4609c642cdae4948f2ea11b5be', 'request_id': 'req-febd5e42-1f06-455c-bbb1-d0e99c489c45'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:06.005530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-febd5e42-1f06-455c-bbb1-d0e99c489c45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0860d4609c642cdae4948f2ea11b5be', 'request_id': 'req-febd5e42-1f06-455c-bbb1-d0e99c489c45'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:06.023549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-febd5e42-1f06-455c-bbb1-d0e99c489c45 admin admin] The token cache is being invalidate because project a0860d4609c642cdae4948f2ea11b5be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:06.023990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-febd5e42-1f06-455c-bbb1-d0e99c489c45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:06.024948 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-febd5e42-1f06-455c-bbb1-d0e99c489c45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:06.026912 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 934/1868] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:05 2022] DELETE /identity/v3/projects/a0860d4609c642cdae4948f2ea11b5be => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:06.032277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2953b-9c53-4e38-add5-ec1a23333785 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.032564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2953b-9c53-4e38-add5-ec1a23333785 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.032839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2953b-9c53-4e38-add5-ec1a23333785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.054297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-ccf2953b-9c53-4e38-add5-ec1a23333785 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:06.055074 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ccf2953b-9c53-4e38-add5-ec1a23333785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:06.112592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:06.124151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 935/1869] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:06.132754 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f682c4-ee60-468f-8e60-38dafbd87992 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.133034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f682c4-ee60-468f-8e60-38dafbd87992 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.133331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f682c4-ee60-468f-8e60-38dafbd87992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.142770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:06.154789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a6f682c4-ee60-468f-8e60-38dafbd87992 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:06.157228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a6f682c4-ee60-468f-8e60-38dafbd87992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:06.165513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 935/1870] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:06.169474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2953b-9c53-4e38-add5-ec1a23333785 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:06.179933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2953b-9c53-4e38-add5-ec1a23333785 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:06.185111 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ad3dd-19d0-4598-9978-7a43e7ffe0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:06.185955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad3dd-19d0-4598-9978-7a43e7ffe0ac admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.186163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad3dd-19d0-4598-9978-7a43e7ffe0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.186427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ad3dd-19d0-4598-9978-7a43e7ffe0ac admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.187083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ad3dd-19d0-4598-9978-7a43e7ffe0ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:06.187385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ad3dd-19d0-4598-9978-7a43e7ffe0ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:06.188976 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ad3dd-19d0-4598-9978-7a43e7ffe0ac admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:06.201133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 936/1871] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:06.205980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f682c4-ee60-468f-8e60-38dafbd87992 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:06.217531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f682c4-ee60-468f-8e60-38dafbd87992 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:06.223291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45632275-ace0-4e54-8720-63dc978b4a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:06.223909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45632275-ace0-4e54-8720-63dc978b4a08 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.224204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45632275-ace0-4e54-8720-63dc978b4a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.224464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45632275-ace0-4e54-8720-63dc978b4a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.225804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45632275-ace0-4e54-8720-63dc978b4a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:06.227204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45632275-ace0-4e54-8720-63dc978b4a08 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:06.262115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 936/1872] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:06 2022] POST /identity/v3/projects => generated 409 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:06.263476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ad3dd-19d0-4598-9978-7a43e7ffe0ac admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:06.274976 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ad3dd-19d0-4598-9978-7a43e7ffe0ac admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:06.336377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad144793-5ca7-44c5-9447-31ebecb80827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:06.337007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144793-5ca7-44c5-9447-31ebecb80827 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.337275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144793-5ca7-44c5-9447-31ebecb80827 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.337538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad144793-5ca7-44c5-9447-31ebecb80827 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.338217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad144793-5ca7-44c5-9447-31ebecb80827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:06.340177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad144793-5ca7-44c5-9447-31ebecb80827 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:06.380860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 937/1873] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:06 2022] POST /identity/v3/users => generated 305 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:06.384878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45632275-ace0-4e54-8720-63dc978b4a08 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:06.396995 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45632275-ace0-4e54-8720-63dc978b4a08 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:06.400557 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad144793-5ca7-44c5-9447-31ebecb80827 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:06.402375 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b564e55-8078-4391-94c2-a702433aae5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:06.402524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ad144793-5ca7-44c5-9447-31ebecb80827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:06.403238 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b564e55-8078-4391-94c2-a702433aae5a admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.403480 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b564e55-8078-4391-94c2-a702433aae5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.403821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b564e55-8078-4391-94c2-a702433aae5a admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.404608 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b564e55-8078-4391-94c2-a702433aae5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:06.406340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b564e55-8078-4391-94c2-a702433aae5a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:06.418476 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 937/1874] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:06.422689 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b564e55-8078-4391-94c2-a702433aae5a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:06.429301 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad144793-5ca7-44c5-9447-31ebecb80827 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:06.435560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356a1be-950d-462d-8d27-9c3a09402d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:06.437475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356a1be-950d-462d-8d27-9c3a09402d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.438379 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356a1be-950d-462d-8d27-9c3a09402d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.439133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356a1be-950d-462d-8d27-9c3a09402d2a admin admin] PATH_INFO: `/v3/projects/5e141024dc66465cb9fc65ce702525d8` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.441031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356a1be-950d-462d-8d27-9c3a09402d2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e141024dc66465cb9fc65ce702525d8)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:06.443068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356a1be-950d-462d-8d27-9c3a09402d2a admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:06.457000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b564e55-8078-4391-94c2-a702433aae5a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:06.462734 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bad9a2-565d-4d44-b0fb-2a5fa6ec1807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:06.463438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bad9a2-565d-4d44-b0fb-2a5fa6ec1807 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.463720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bad9a2-565d-4d44-b0fb-2a5fa6ec1807 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.464009 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bad9a2-565d-4d44-b0fb-2a5fa6ec1807 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.464734 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bad9a2-565d-4d44-b0fb-2a5fa6ec1807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:06.466926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bad9a2-565d-4d44-b0fb-2a5fa6ec1807 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:06.481306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 938/1875] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:06.491253 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bad9a2-565d-4d44-b0fb-2a5fa6ec1807 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:06.507599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b356a1be-950d-462d-8d27-9c3a09402d2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e141024dc66465cb9fc65ce702525d8', 'request_id': 'req-b356a1be-950d-462d-8d27-9c3a09402d2a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:06.515714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bad9a2-565d-4d44-b0fb-2a5fa6ec1807 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:06.522272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b356a1be-950d-462d-8d27-9c3a09402d2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e141024dc66465cb9fc65ce702525d8', 'request_id': 'req-b356a1be-950d-462d-8d27-9c3a09402d2a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:06.533387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b356a1be-950d-462d-8d27-9c3a09402d2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e141024dc66465cb9fc65ce702525d8', 'request_id': 'req-b356a1be-950d-462d-8d27-9c3a09402d2a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:06.546215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b356a1be-950d-462d-8d27-9c3a09402d2a admin admin] The token cache is being invalidate because project 5e141024dc66465cb9fc65ce702525d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:06.546655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b356a1be-950d-462d-8d27-9c3a09402d2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:06.547428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b356a1be-950d-462d-8d27-9c3a09402d2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:06.549295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 938/1876] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:06 2022] DELETE /identity/v3/projects/5e141024dc66465cb9fc65ce702525d8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:06.554871 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4043d0-bc81-4506-bd6e-3a9f24cf1cc7 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.554871 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4043d0-bc81-4506-bd6e-3a9f24cf1cc7 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.555255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4043d0-bc81-4506-bd6e-3a9f24cf1cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.590568 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1c4043d0-bc81-4506-bd6e-3a9f24cf1cc7 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:06.592052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1c4043d0-bc81-4506-bd6e-3a9f24cf1cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:06.609322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aae12d-77d0-4ecf-96d8-2ec8dff61048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:06.611346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aae12d-77d0-4ecf-96d8-2ec8dff61048 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.611698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aae12d-77d0-4ecf-96d8-2ec8dff61048 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.612035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aae12d-77d0-4ecf-96d8-2ec8dff61048 admin admin] PATH_INFO: `/v3/projects/fc5175fe08c240a9b090258cf832294c/users/8406fab60cb4464996b8d929da311f89/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.636846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aae12d-77d0-4ecf-96d8-2ec8dff61048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc5175fe08c240a9b090258cf832294c, user_id=8406fab60cb4464996b8d929da311f89, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:06.644799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aae12d-77d0-4ecf-96d8-2ec8dff61048 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:06.663432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 939/1877] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:06 2022] PUT /identity/v3/projects/fc5175fe08c240a9b090258cf832294c/users/8406fab60cb4464996b8d929da311f89/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:06.675950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b9bca-ab75-47f4-9c98-c1676cac13ed None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.676434 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b9bca-ab75-47f4-9c98-c1676cac13ed None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.676851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b9bca-ab75-47f4-9c98-c1676cac13ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.685612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:06.698171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 939/1878] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:06.706102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98134fc1-2050-4e0b-abd7-dabf5ebbec95 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.706687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98134fc1-2050-4e0b-abd7-dabf5ebbec95 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.707036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98134fc1-2050-4e0b-abd7-dabf5ebbec95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.729243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7d3b9bca-ab75-47f4-9c98-c1676cac13ed None None] MFA Rules not processed for user `8406fab60cb4464996b8d929da311f89`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:06.730611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7d3b9bca-ab75-47f4-9c98-c1676cac13ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:06.733015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-98134fc1-2050-4e0b-abd7-dabf5ebbec95 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:06.734796 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-98134fc1-2050-4e0b-abd7-dabf5ebbec95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:06.804061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 940/1879] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:06.805074 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98134fc1-2050-4e0b-abd7-dabf5ebbec95 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:06.816437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98134fc1-2050-4e0b-abd7-dabf5ebbec95 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:06.820816 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 940/1880] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:06 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:06.825037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b9bca-ab75-47f4-9c98-c1676cac13ed None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:06.825871 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442339b8-5052-4fc3-a888-13ee12566d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:06.826399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7d3b9bca-ab75-47f4-9c98-c1676cac13ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:06.826514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442339b8-5052-4fc3-a888-13ee12566d25 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.826768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442339b8-5052-4fc3-a888-13ee12566d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.828053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442339b8-5052-4fc3-a888-13ee12566d25 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.828814 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442339b8-5052-4fc3-a888-13ee12566d25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:06.829118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442339b8-5052-4fc3-a888-13ee12566d25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:06.830940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442339b8-5052-4fc3-a888-13ee12566d25 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:06.840972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b9bca-ab75-47f4-9c98-c1676cac13ed None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:06.845486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 941/1881] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:53:06.847075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83513e-f917-4dff-8942-3a852b81e755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:06.847694 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83513e-f917-4dff-8942-3a852b81e755 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.848085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83513e-f917-4dff-8942-3a852b81e755 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.848367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83513e-f917-4dff-8942-3a852b81e755 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.849427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83513e-f917-4dff-8942-3a852b81e755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:06.850798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442339b8-5052-4fc3-a888-13ee12566d25 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:06.851438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83513e-f917-4dff-8942-3a852b81e755 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:06.863411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442339b8-5052-4fc3-a888-13ee12566d25 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:06.877055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09c846-1cda-42f1-b9aa-f0d5fc860b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:06.878493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09c846-1cda-42f1-b9aa-f0d5fc860b9b admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:06.878786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09c846-1cda-42f1-b9aa-f0d5fc860b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:06.879058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09c846-1cda-42f1-b9aa-f0d5fc860b9b admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:06.879862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09c846-1cda-42f1-b9aa-f0d5fc860b9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:06.883129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf09c846-1cda-42f1-b9aa-f0d5fc860b9b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:06.894711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 941/1882] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:06 2022] POST /identity/v3/projects => generated 407 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:06.900175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83513e-f917-4dff-8942-3a852b81e755 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:06.912241 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:06.935845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 942/1883] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:06 2022] POST /identity/v3/projects => generated 379 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:06.940164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09c846-1cda-42f1-b9aa-f0d5fc860b9b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.004969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83513e-f917-4dff-8942-3a852b81e755 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.011770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09c846-1cda-42f1-b9aa-f0d5fc860b9b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.015725 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ecfa7-e107-49b8-b9e4-f9766664431e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.017018 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ecfa7-e107-49b8-b9e4-f9766664431e admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.017330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ecfa7-e107-49b8-b9e4-f9766664431e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.017612 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ecfa7-e107-49b8-b9e4-f9766664431e admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.018346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ecfa7-e107-49b8-b9e4-f9766664431e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.020070 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa563e-f53b-419b-8b44-0499058175b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.020377 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ecfa7-e107-49b8-b9e4-f9766664431e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.020897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa563e-f53b-419b-8b44-0499058175b4 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.021304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa563e-f53b-419b-8b44-0499058175b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.021644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa563e-f53b-419b-8b44-0499058175b4 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.022370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa563e-f53b-419b-8b44-0499058175b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.024699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa563e-f53b-419b-8b44-0499058175b4 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.046901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 942/1884] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:06 2022] POST /identity/v3/users => generated 310 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:07.053596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 943/1885] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:06 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:07.053768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ecfa7-e107-49b8-b9e4-f9766664431e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.054665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa563e-f53b-419b-8b44-0499058175b4 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.063973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ecfa7-e107-49b8-b9e4-f9766664431e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.065752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa563e-f53b-419b-8b44-0499058175b4 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.068821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd80b71-3134-48d8-855c-febaebf0c683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.069541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd80b71-3134-48d8-855c-febaebf0c683 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.069948 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd80b71-3134-48d8-855c-febaebf0c683 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.070308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd80b71-3134-48d8-855c-febaebf0c683 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.070643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792c119-cb5c-472f-a50c-a2978f2987b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.071069 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd80b71-3134-48d8-855c-febaebf0c683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.071242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792c119-cb5c-472f-a50c-a2978f2987b9 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.071672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792c119-cb5c-472f-a50c-a2978f2987b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.071955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792c119-cb5c-472f-a50c-a2978f2987b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.072604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792c119-cb5c-472f-a50c-a2978f2987b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.072966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd80b71-3134-48d8-855c-febaebf0c683 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.074621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792c119-cb5c-472f-a50c-a2978f2987b9 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.084001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 943/1886] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:07.088189 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd80b71-3134-48d8-855c-febaebf0c683 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.091212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 944/1887] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:07.094172 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792c119-cb5c-472f-a50c-a2978f2987b9 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.098707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd80b71-3134-48d8-855c-febaebf0c683 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.106642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071ad66-89c9-4174-a117-8625b83eab97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.107251 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792c119-cb5c-472f-a50c-a2978f2987b9 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.107382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071ad66-89c9-4174-a117-8625b83eab97 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.107581 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071ad66-89c9-4174-a117-8625b83eab97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.107855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071ad66-89c9-4174-a117-8625b83eab97 admin admin] PATH_INFO: `/v3/projects/af8a9ff7e3bd4df1bec68a1fc346a311/users/48e0fa0c29d845bc850b8c9400fc8f7f/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.112332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bb9a9-0138-420a-a768-cdf56a500fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.113151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bb9a9-0138-420a-a768-cdf56a500fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.113602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bb9a9-0138-420a-a768-cdf56a500fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.113996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bb9a9-0138-420a-a768-cdf56a500fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.114743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bb9a9-0138-420a-a768-cdf56a500fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.117061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bb9a9-0138-420a-a768-cdf56a500fd3 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.123378 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2071ad66-89c9-4174-a117-8625b83eab97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af8a9ff7e3bd4df1bec68a1fc346a311, user_id=48e0fa0c29d845bc850b8c9400fc8f7f, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.126191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2071ad66-89c9-4174-a117-8625b83eab97 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.131234 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 945/1888] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:07.135797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bb9a9-0138-420a-a768-cdf56a500fd3 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.141540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 944/1889] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:07 2022] PUT /identity/v3/projects/af8a9ff7e3bd4df1bec68a1fc346a311/users/48e0fa0c29d845bc850b8c9400fc8f7f/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:07.145448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071ad66-89c9-4174-a117-8625b83eab97 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.147801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bb9a9-0138-420a-a768-cdf56a500fd3 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.183888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:07.214911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071ad66-89c9-4174-a117-8625b83eab97 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.215481 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701912f-eba5-454e-8e5f-c2c6b075d35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.216143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701912f-eba5-454e-8e5f-c2c6b075d35c admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.216416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701912f-eba5-454e-8e5f-c2c6b075d35c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.216683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701912f-eba5-454e-8e5f-c2c6b075d35c admin admin] PATH_INFO: `/v3/projects/f2fae35cf21340a3b90015ee4c184044/users/e41f21bcd76c4091b32c604a494bd8e9/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.220283 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84625f2-68e9-49bb-9ce2-33836381210f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.220893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84625f2-68e9-49bb-9ce2-33836381210f admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.221172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84625f2-68e9-49bb-9ce2-33836381210f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.221442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84625f2-68e9-49bb-9ce2-33836381210f admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.222110 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84625f2-68e9-49bb-9ce2-33836381210f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.224013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84625f2-68e9-49bb-9ce2-33836381210f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.231709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701912f-eba5-454e-8e5f-c2c6b075d35c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2fae35cf21340a3b90015ee4c184044, user_id=e41f21bcd76c4091b32c604a494bd8e9, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.234179 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 945/1890] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:07.237611 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701912f-eba5-454e-8e5f-c2c6b075d35c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.238136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84625f2-68e9-49bb-9ce2-33836381210f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.249454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84625f2-68e9-49bb-9ce2-33836381210f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.251741 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 946/1891] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:07 2022] PUT /identity/v3/projects/f2fae35cf21340a3b90015ee4c184044/users/e41f21bcd76c4091b32c604a494bd8e9/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:07.262725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643be323-bb3f-4ca3-bac0-11640d5d60eb None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.262975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643be323-bb3f-4ca3-bac0-11640d5d60eb None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.263250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643be323-bb3f-4ca3-bac0-11640d5d60eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.291125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-643be323-bb3f-4ca3-bac0-11640d5d60eb None None] MFA Rules not processed for user `e41f21bcd76c4091b32c604a494bd8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:07.292322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-643be323-bb3f-4ca3-bac0-11640d5d60eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:07.303699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:07.312079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1871c-47ea-4aab-974a-a83d7488ce41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.312750 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1871c-47ea-4aab-974a-a83d7488ce41 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.313030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1871c-47ea-4aab-974a-a83d7488ce41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.313302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1871c-47ea-4aab-974a-a83d7488ce41 admin admin] PATH_INFO: `/v3/projects/af8a9ff7e3bd4df1bec68a1fc346a311/users/48e0fa0c29d845bc850b8c9400fc8f7f/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.315375 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1871c-47ea-4aab-974a-a83d7488ce41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af8a9ff7e3bd4df1bec68a1fc346a311, user_id=48e0fa0c29d845bc850b8c9400fc8f7f, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.318044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1871c-47ea-4aab-974a-a83d7488ce41 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.329293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 946/1892] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:07 2022] PUT /identity/v3/projects/af8a9ff7e3bd4df1bec68a1fc346a311/users/48e0fa0c29d845bc850b8c9400fc8f7f/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:07.339816 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795d6c7-c8ca-41a0-9ace-e12fdda38b5a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.340182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795d6c7-c8ca-41a0-9ace-e12fdda38b5a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.340525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795d6c7-c8ca-41a0-9ace-e12fdda38b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.361431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 947/1893] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:07.365797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643be323-bb3f-4ca3-bac0-11640d5d60eb None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.372675 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7795d6c7-c8ca-41a0-9ace-e12fdda38b5a None None] MFA Rules not processed for user `48e0fa0c29d845bc850b8c9400fc8f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:07.374430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7795d6c7-c8ca-41a0-9ace-e12fdda38b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:07.377780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643be323-bb3f-4ca3-bac0-11640d5d60eb None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.442009 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6eb73-e9be-4e2a-b09d-796abbf45cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.442009 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6eb73-e9be-4e2a-b09d-796abbf45cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.442009 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6eb73-e9be-4e2a-b09d-796abbf45cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.442009 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6eb73-e9be-4e2a-b09d-796abbf45cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.442433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6eb73-e9be-4e2a-b09d-796abbf45cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.443379 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6eb73-e9be-4e2a-b09d-796abbf45cb5 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.464194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 947/1894] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:07 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:07.471926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795d6c7-c8ca-41a0-9ace-e12fdda38b5a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.473509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7795d6c7-c8ca-41a0-9ace-e12fdda38b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:07.480878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 948/1895] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:07 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:07.481915 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6eb73-e9be-4e2a-b09d-796abbf45cb5 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.523719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:07.542810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795d6c7-c8ca-41a0-9ace-e12fdda38b5a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.555593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6eb73-e9be-4e2a-b09d-796abbf45cb5 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.560747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c03316-8152-4ca1-9ba7-1ad29d7e7224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.562032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c03316-8152-4ca1-9ba7-1ad29d7e7224 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.562255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c03316-8152-4ca1-9ba7-1ad29d7e7224 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.562511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c03316-8152-4ca1-9ba7-1ad29d7e7224 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.563191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c03316-8152-4ca1-9ba7-1ad29d7e7224 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.565110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c03316-8152-4ca1-9ba7-1ad29d7e7224 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.589353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 949/1896] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:07 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:07.593350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c03316-8152-4ca1-9ba7-1ad29d7e7224 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.597632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079293c-6361-4da5-a066-fc6fecde6473 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.598353 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079293c-6361-4da5-a066-fc6fecde6473 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.598744 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079293c-6361-4da5-a066-fc6fecde6473 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.599512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079293c-6361-4da5-a066-fc6fecde6473 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.602756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c03316-8152-4ca1-9ba7-1ad29d7e7224 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.607477 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ef7c1-4657-42b8-bf1e-54e3cf64ac0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.608059 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ef7c1-4657-42b8-bf1e-54e3cf64ac0b admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.608364 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ef7c1-4657-42b8-bf1e-54e3cf64ac0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.608643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ef7c1-4657-42b8-bf1e-54e3cf64ac0b admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.609258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ef7c1-4657-42b8-bf1e-54e3cf64ac0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.610855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ef7c1-4657-42b8-bf1e-54e3cf64ac0b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.620853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 950/1897] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:07.625252 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ef7c1-4657-42b8-bf1e-54e3cf64ac0b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.634673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ef7c1-4657-42b8-bf1e-54e3cf64ac0b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.640981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a937605-e957-4dc8-bcfa-92e95246e2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.641713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a937605-e957-4dc8-bcfa-92e95246e2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.642125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a937605-e957-4dc8-bcfa-92e95246e2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.642497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a937605-e957-4dc8-bcfa-92e95246e2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.643295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a937605-e957-4dc8-bcfa-92e95246e2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.645053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a937605-e957-4dc8-bcfa-92e95246e2b8 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.651581 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:07.660873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 951/1898] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:07.662046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a937605-e957-4dc8-bcfa-92e95246e2b8 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.672311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a937605-e957-4dc8-bcfa-92e95246e2b8 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.677485 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d28aa-dea4-4d0b-9780-3d372ce218b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.678103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079293c-6361-4da5-a066-fc6fecde6473 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.678230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d28aa-dea4-4d0b-9780-3d372ce218b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.678457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d28aa-dea4-4d0b-9780-3d372ce218b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.678784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d28aa-dea4-4d0b-9780-3d372ce218b1 admin admin] PATH_INFO: `/v3/projects/5d3b922f1b11426db1f5ce6a20beca57/users/bf131e3e2d3b47f09ae4cfa83fdd60f8/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.679684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079293c-6361-4da5-a066-fc6fecde6473 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.690332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 948/1899] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:07 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:07.699920 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d28aa-dea4-4d0b-9780-3d372ce218b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d3b922f1b11426db1f5ce6a20beca57, user_id=bf131e3e2d3b47f09ae4cfa83fdd60f8, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.703321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d28aa-dea4-4d0b-9780-3d372ce218b1 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.718533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 952/1900] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:07 2022] PUT /identity/v3/projects/5d3b922f1b11426db1f5ce6a20beca57/users/bf131e3e2d3b47f09ae4cfa83fdd60f8/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:07.729776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb52f9-565d-40bf-9d13-c2a36deb97fb None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.730197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb52f9-565d-40bf-9d13-c2a36deb97fb None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.730587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb52f9-565d-40bf-9d13-c2a36deb97fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.778118 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-63fb52f9-565d-40bf-9d13-c2a36deb97fb None None] MFA Rules not processed for user `bf131e3e2d3b47f09ae4cfa83fdd60f8`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:07.779515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-63fb52f9-565d-40bf-9d13-c2a36deb97fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:07.859748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 949/1901] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:07.862261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d28aa-dea4-4d0b-9780-3d372ce218b1 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.903579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb52f9-565d-40bf-9d13-c2a36deb97fb None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:07.961235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d28aa-dea4-4d0b-9780-3d372ce218b1 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.966312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02a453-7ef0-4fd4-a2ca-fd716f7a646d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.966930 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02a453-7ef0-4fd4-a2ca-fd716f7a646d admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.967206 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02a453-7ef0-4fd4-a2ca-fd716f7a646d admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.967477 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02a453-7ef0-4fd4-a2ca-fd716f7a646d admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:07.968170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f02a453-7ef0-4fd4-a2ca-fd716f7a646d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:07.970083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f02a453-7ef0-4fd4-a2ca-fd716f7a646d admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:07.992916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb52f9-565d-40bf-9d13-c2a36deb97fb None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:07.996758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eae84-ddcb-4cea-af0b-24709c38c77d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:07.997374 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eae84-ddcb-4cea-af0b-24709c38c77d service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:07.997654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eae84-ddcb-4cea-af0b-24709c38c77d service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:07.997960 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eae84-ddcb-4cea-af0b-24709c38c77d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:08.000320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 953/1902] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:07 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:08.004457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02a453-7ef0-4fd4-a2ca-fd716f7a646d admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:08.061621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604eae84-ddcb-4cea-af0b-24709c38c77d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:08.063128 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604eae84-ddcb-4cea-af0b-24709c38c77d service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:08.073168 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 950/1903] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:07 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:08.076830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02a453-7ef0-4fd4-a2ca-fd716f7a646d admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:08.081650 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881065a-642f-4efe-85f0-875f934edb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:08.082610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881065a-642f-4efe-85f0-875f934edb9c admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:08.082889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881065a-642f-4efe-85f0-875f934edb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:08.083163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881065a-642f-4efe-85f0-875f934edb9c admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:08.083800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881065a-642f-4efe-85f0-875f934edb9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:08.086003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881065a-642f-4efe-85f0-875f934edb9c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:08.096619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:08.116635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 954/1904] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:08 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:08.120721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eae84-ddcb-4cea-af0b-24709c38c77d service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:08.132609 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eae84-ddcb-4cea-af0b-24709c38c77d service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:08.137513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de4e4e-13a5-49f8-8a09-9e04ce55031d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:08.138147 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de4e4e-13a5-49f8-8a09-9e04ce55031d admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:08.138426 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de4e4e-13a5-49f8-8a09-9e04ce55031d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:08.138696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de4e4e-13a5-49f8-8a09-9e04ce55031d admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:08.139344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de4e4e-13a5-49f8-8a09-9e04ce55031d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:08.141053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de4e4e-13a5-49f8-8a09-9e04ce55031d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:08.151523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 951/1905] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:08.156795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881065a-642f-4efe-85f0-875f934edb9c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:08.167135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881065a-642f-4efe-85f0-875f934edb9c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:08.172502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e4bd4-e106-439a-ac94-3bbe095d491d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:08.173169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e4bd4-e106-439a-ac94-3bbe095d491d admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:08.174104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e4bd4-e106-439a-ac94-3bbe095d491d admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:08.174104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e4bd4-e106-439a-ac94-3bbe095d491d admin admin] PATH_INFO: `/v3/projects/97c64dd8363b4450b3400e94fc531483/users/b1a1af4b9be2416aa77283fe3bbfe0b6/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:08.192141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e4bd4-e106-439a-ac94-3bbe095d491d admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c64dd8363b4450b3400e94fc531483, user_id=b1a1af4b9be2416aa77283fe3bbfe0b6, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:08.195638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e4bd4-e106-439a-ac94-3bbe095d491d admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:08.229712 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 955/1906] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:08 2022] PUT /identity/v3/projects/97c64dd8363b4450b3400e94fc531483/users/b1a1af4b9be2416aa77283fe3bbfe0b6/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:08.234707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de4e4e-13a5-49f8-8a09-9e04ce55031d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:08.245287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de4e4e-13a5-49f8-8a09-9e04ce55031d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:08.259599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:08.331992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf11576-523c-4471-9ffb-dc41e7b0ee72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:08.332614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf11576-523c-4471-9ffb-dc41e7b0ee72 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:08.332899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf11576-523c-4471-9ffb-dc41e7b0ee72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:08.333174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf11576-523c-4471-9ffb-dc41e7b0ee72 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:08.333957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf11576-523c-4471-9ffb-dc41e7b0ee72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:08.335574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf11576-523c-4471-9ffb-dc41e7b0ee72 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:08.349768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 952/1907] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:08 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:08.352524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e4bd4-e106-439a-ac94-3bbe095d491d admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:08.365471 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e4bd4-e106-439a-ac94-3bbe095d491d admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:08.370423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d2d06-554f-47a1-8124-a5d7f543d359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:08.371202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d2d06-554f-47a1-8124-a5d7f543d359 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:08.371481 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d2d06-554f-47a1-8124-a5d7f543d359 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:08.371753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d2d06-554f-47a1-8124-a5d7f543d359 admin admin] PATH_INFO: `/v3/projects/97c64dd8363b4450b3400e94fc531483/users/b1a1af4b9be2416aa77283fe3bbfe0b6/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:08.373927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d2d06-554f-47a1-8124-a5d7f543d359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c64dd8363b4450b3400e94fc531483, user_id=b1a1af4b9be2416aa77283fe3bbfe0b6, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:08.390469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d2d06-554f-47a1-8124-a5d7f543d359 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:08.402334 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 956/1908] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:08 2022] PUT /identity/v3/projects/97c64dd8363b4450b3400e94fc531483/users/b1a1af4b9be2416aa77283fe3bbfe0b6/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:08.412323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82006e-c7d2-4378-b368-fad556c07cce None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:08.412615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82006e-c7d2-4378-b368-fad556c07cce None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:08.412923 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82006e-c7d2-4378-b368-fad556c07cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:08.461457 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-de82006e-c7d2-4378-b368-fad556c07cce None None] MFA Rules not processed for user `b1a1af4b9be2416aa77283fe3bbfe0b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:08.462798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-de82006e-c7d2-4378-b368-fad556c07cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:08.533450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 953/1909] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:08.545020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d2d06-554f-47a1-8124-a5d7f543d359 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:08.603172 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d2d06-554f-47a1-8124-a5d7f543d359 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:08.607016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae8cb8-c17c-405a-9f59-1361943fb8b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:08.607799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae8cb8-c17c-405a-9f59-1361943fb8b1 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:08.608082 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae8cb8-c17c-405a-9f59-1361943fb8b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:08.608384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ae8cb8-c17c-405a-9f59-1361943fb8b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:08.618782 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae8cb8-c17c-405a-9f59-1361943fb8b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:08.620204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ae8cb8-c17c-405a-9f59-1361943fb8b1 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:08.630290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 957/1910] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:08.675265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82006e-c7d2-4378-b368-fad556c07cce None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:08.677214 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-de82006e-c7d2-4378-b368-fad556c07cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:08.689541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82006e-c7d2-4378-b368-fad556c07cce None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:08.693185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5413a75-9abe-454f-984d-34a891478a9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:08.693841 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5413a75-9abe-454f-984d-34a891478a9b service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:08.694130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5413a75-9abe-454f-984d-34a891478a9b service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:08.694389 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5413a75-9abe-454f-984d-34a891478a9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:08.759392 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5413a75-9abe-454f-984d-34a891478a9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:08.760857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5413a75-9abe-454f-984d-34a891478a9b service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:08.770516 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 954/1911] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:11.125694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae8cb8-c17c-405a-9f59-1361943fb8b1 service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:11.128340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-19ae8cb8-c17c-405a-9f59-1361943fb8b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:11.245933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ae8cb8-c17c-405a-9f59-1361943fb8b1 service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:11.259474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:11.272347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:11.272648 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:11.272927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] PATH_INFO: `/v3/users/9ab658a6718943fea13aae3eb0330247` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:11.288847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab658a6718943fea13aae3eb0330247)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:11.292784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:11.395758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab658a6718943fea13aae3eb0330247'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:11.404698 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab658a6718943fea13aae3eb0330247'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:11.405585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab658a6718943fea13aae3eb0330247'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:11.406325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab658a6718943fea13aae3eb0330247'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:11.409307 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:INSERT=6,keystone:SELECT=86,keystone:DELETE=8 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:11.417841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab658a6718943fea13aae3eb0330247'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:11.453542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 958/1912] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:11 2022] DELETE /identity/v3/users/9ab658a6718943fea13aae3eb0330247 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:11.458266 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe5561-7977-4b67-ae89-9cab2e0cc081 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:11.458787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe5561-7977-4b67-ae89-9cab2e0cc081 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:11.458907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe5561-7977-4b67-ae89-9cab2e0cc081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:11.482318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-09fe5561-7977-4b67-ae89-9cab2e0cc081 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:11.484051 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-09fe5561-7977-4b67-ae89-9cab2e0cc081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:11.526904 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:11.560702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 955/1913] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:11.571211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:11.572763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:11.635185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b905c5-b368-4ea6-81bb-ba5416452d9c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:11.641214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9d19e-dc44-443d-8b8e-84fbd8b4f51b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:11.642149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9d19e-dc44-443d-8b8e-84fbd8b4f51b service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:11.642542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9d19e-dc44-443d-8b8e-84fbd8b4f51b service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:11.642911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9d19e-dc44-443d-8b8e-84fbd8b4f51b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:11.656260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9d19e-dc44-443d-8b8e-84fbd8b4f51b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:11.657706 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9d19e-dc44-443d-8b8e-84fbd8b4f51b service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:11.668936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 959/1914] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:11.967018 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe5561-7977-4b67-ae89-9cab2e0cc081 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:11.968980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-09fe5561-7977-4b67-ae89-9cab2e0cc081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:11.988407 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe5561-7977-4b67-ae89-9cab2e0cc081 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:11.995157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:11.995819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:11.996098 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:11.996372 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] PATH_INFO: `/v3/projects/4a0dc70d5a194b3a958ef5dfcab44533` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:11.997531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a0dc70d5a194b3a958ef5dfcab44533)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:11.999860 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:12.060699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a0dc70d5a194b3a958ef5dfcab44533', 'request_id': 'req-c59a40b2-368b-4d5f-bf58-726b40271a3c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.068244 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a0dc70d5a194b3a958ef5dfcab44533', 'request_id': 'req-c59a40b2-368b-4d5f-bf58-726b40271a3c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.074360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a0dc70d5a194b3a958ef5dfcab44533', 'request_id': 'req-c59a40b2-368b-4d5f-bf58-726b40271a3c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.084839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9d19e-dc44-443d-8b8e-84fbd8b4f51b service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:12.086616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e2e9d19e-dc44-443d-8b8e-84fbd8b4f51b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:12.095322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] The token cache is being invalidate because project 4a0dc70d5a194b3a958ef5dfcab44533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:12.095619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.096272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.100752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 956/1915] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:11 2022] DELETE /identity/v3/projects/4a0dc70d5a194b3a958ef5dfcab44533 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:12.100887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9d19e-dc44-443d-8b8e-84fbd8b4f51b service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:12.101764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:12.103107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:12.160243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f9210-5ad2-4a0a-9ba6-a4dd1de725a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:12.160795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f9210-5ad2-4a0a-9ba6-a4dd1de725a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:12.161036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f9210-5ad2-4a0a-9ba6-a4dd1de725a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:12.161305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f9210-5ad2-4a0a-9ba6-a4dd1de725a1 admin admin] PATH_INFO: `/v3/users/e41f21bcd76c4091b32c604a494bd8e9` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:12.162434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f9210-5ad2-4a0a-9ba6-a4dd1de725a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e41f21bcd76c4091b32c604a494bd8e9)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:12.164361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f9210-5ad2-4a0a-9ba6-a4dd1de725a1 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:12.166570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a40b2-368b-4d5f-bf58-726b40271a3c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:12.171821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:12.172454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:12.172738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:12.173008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] PATH_INFO: `/v3/users/a88f709b48034fb3809d435869398b02` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:12.174177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a88f709b48034fb3809d435869398b02)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:12.176101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:12.214852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-977f9210-5ad2-4a0a-9ba6-a4dd1de725a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e41f21bcd76c4091b32c604a494bd8e9'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.222923 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-977f9210-5ad2-4a0a-9ba6-a4dd1de725a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e41f21bcd76c4091b32c604a494bd8e9'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.223608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-977f9210-5ad2-4a0a-9ba6-a4dd1de725a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e41f21bcd76c4091b32c604a494bd8e9'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.224260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-977f9210-5ad2-4a0a-9ba6-a4dd1de725a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e41f21bcd76c4091b32c604a494bd8e9'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.225476 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a88f709b48034fb3809d435869398b02'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.231793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-977f9210-5ad2-4a0a-9ba6-a4dd1de725a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e41f21bcd76c4091b32c604a494bd8e9'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.233385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a88f709b48034fb3809d435869398b02'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.234367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a88f709b48034fb3809d435869398b02'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.234989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a88f709b48034fb3809d435869398b02'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.240050 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:12.244072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a88f709b48034fb3809d435869398b02'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.244251 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:12.254304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 960/1916] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:12 2022] DELETE /identity/v3/users/e41f21bcd76c4091b32c604a494bd8e9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:12.258804 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406053-cfbf-4b77-9658-3d86286bfa45 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:12.259096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406053-cfbf-4b77-9658-3d86286bfa45 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:12.259373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa406053-cfbf-4b77-9658-3d86286bfa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:12.263376 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 957/1917] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:12 2022] DELETE /identity/v3/users/a88f709b48034fb3809d435869398b02 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:12.278999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-fa406053-cfbf-4b77-9658-3d86286bfa45 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:12.281090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-fa406053-cfbf-4b77-9658-3d86286bfa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:12.367673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 961/1918] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:12.381053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:12.382549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:12.462857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee228b3-aeca-4db3-afd4-15cb3ab077d8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:12.466949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c9e46-8335-437f-ad68-294a610b06f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:12.468162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c9e46-8335-437f-ad68-294a610b06f7 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:12.468459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c9e46-8335-437f-ad68-294a610b06f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:12.468730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c9e46-8335-437f-ad68-294a610b06f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:12.479527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c9e46-8335-437f-ad68-294a610b06f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:12.481042 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c9e46-8335-437f-ad68-294a610b06f7 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:12.491630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 958/1919] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:12.640515 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa406053-cfbf-4b77-9658-3d86286bfa45 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:12.641980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-fa406053-cfbf-4b77-9658-3d86286bfa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:12.656236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa406053-cfbf-4b77-9658-3d86286bfa45 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:12.663984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50651cc6-99ea-48ff-93b1-e7e2ddd63236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:12.664651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50651cc6-99ea-48ff-93b1-e7e2ddd63236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:12.664930 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50651cc6-99ea-48ff-93b1-e7e2ddd63236 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:12.665244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50651cc6-99ea-48ff-93b1-e7e2ddd63236 admin admin] PATH_INFO: `/v3/projects/d88a346a96a94eb89843dbcf7664ca00` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:12.672606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50651cc6-99ea-48ff-93b1-e7e2ddd63236 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d88a346a96a94eb89843dbcf7664ca00)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:12.675487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50651cc6-99ea-48ff-93b1-e7e2ddd63236 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:12.759416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-50651cc6-99ea-48ff-93b1-e7e2ddd63236 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd88a346a96a94eb89843dbcf7664ca00', 'request_id': 'req-50651cc6-99ea-48ff-93b1-e7e2ddd63236'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.766177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-50651cc6-99ea-48ff-93b1-e7e2ddd63236 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd88a346a96a94eb89843dbcf7664ca00', 'request_id': 'req-50651cc6-99ea-48ff-93b1-e7e2ddd63236'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.775076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-50651cc6-99ea-48ff-93b1-e7e2ddd63236 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd88a346a96a94eb89843dbcf7664ca00', 'request_id': 'req-50651cc6-99ea-48ff-93b1-e7e2ddd63236'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.786860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-50651cc6-99ea-48ff-93b1-e7e2ddd63236 admin admin] The token cache is being invalidate because project d88a346a96a94eb89843dbcf7664ca00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:12.787149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-50651cc6-99ea-48ff-93b1-e7e2ddd63236 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.787797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-50651cc6-99ea-48ff-93b1-e7e2ddd63236 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:12.789302 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 962/1920] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:12 2022] DELETE /identity/v3/projects/d88a346a96a94eb89843dbcf7664ca00 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:12.794122 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62819f61-5cc4-401f-b5a6-8d1b8867bac9 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:12.794413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62819f61-5cc4-401f-b5a6-8d1b8867bac9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:12.794688 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62819f61-5cc4-401f-b5a6-8d1b8867bac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:12.816153 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-62819f61-5cc4-401f-b5a6-8d1b8867bac9 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:12.817352 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-62819f61-5cc4-401f-b5a6-8d1b8867bac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:12.881990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 959/1921] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:12.889883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfed282-bc31-4185-8d4b-d08551aeb847 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:12.890172 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfed282-bc31-4185-8d4b-d08551aeb847 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:12.890456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfed282-bc31-4185-8d4b-d08551aeb847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:12.916293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-fdfed282-bc31-4185-8d4b-d08551aeb847 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:12.917512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-fdfed282-bc31-4185-8d4b-d08551aeb847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:12.926229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 963/1922] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:12.930652 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62819f61-5cc4-401f-b5a6-8d1b8867bac9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:12.940424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62819f61-5cc4-401f-b5a6-8d1b8867bac9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:12.944943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50da2-a025-431c-b989-9c270684a5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:12.945621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50da2-a025-431c-b989-9c270684a5c6 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:12.945984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50da2-a025-431c-b989-9c270684a5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:12.946318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50da2-a025-431c-b989-9c270684a5c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:12.947180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe50da2-a025-431c-b989-9c270684a5c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:12.947485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe50da2-a025-431c-b989-9c270684a5c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:12.949065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe50da2-a025-431c-b989-9c270684a5c6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:12.965292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 960/1923] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:12.966442 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfed282-bc31-4185-8d4b-d08551aeb847 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:12.976582 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfed282-bc31-4185-8d4b-d08551aeb847 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:12.982410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d084-6d2a-4ab8-b669-fc537c17518a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:12.983054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d084-6d2a-4ab8-b669-fc537c17518a admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:12.983855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d084-6d2a-4ab8-b669-fc537c17518a admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:12.984257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d084-6d2a-4ab8-b669-fc537c17518a admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:12.984896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6d084-6d2a-4ab8-b669-fc537c17518a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:12.986831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6d084-6d2a-4ab8-b669-fc537c17518a admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:13.033688 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50da2-a025-431c-b989-9c270684a5c6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:13.034847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 964/1924] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:12 2022] POST /identity/v3/projects => generated 365 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:13.035436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ebe50da2-a025-431c-b989-9c270684a5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:13.042593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d084-6d2a-4ab8-b669-fc537c17518a admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:13.053931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50da2-a025-431c-b989-9c270684a5c6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:13.101187 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:13.115046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d084-6d2a-4ab8-b669-fc537c17518a admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:13.119122 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:13.119784 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.119933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5a6ca-607b-4a87-8b8e-c5a90b2fc4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:13.120185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.120467 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] PATH_INFO: `/v3/projects/f2fae35cf21340a3b90015ee4c184044` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.120558 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5a6ca-607b-4a87-8b8e-c5a90b2fc4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.120807 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5a6ca-607b-4a87-8b8e-c5a90b2fc4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.121082 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a5a6ca-607b-4a87-8b8e-c5a90b2fc4f5 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.121594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2fae35cf21340a3b90015ee4c184044)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:13.121713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5a6ca-607b-4a87-8b8e-c5a90b2fc4f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:13.123539 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:13.123649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a5a6ca-607b-4a87-8b8e-c5a90b2fc4f5 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:13.131846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:13.146576 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 965/1925] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:13 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:13.150751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5a6ca-607b-4a87-8b8e-c5a90b2fc4f5 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:13.162273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a5a6ca-607b-4a87-8b8e-c5a90b2fc4f5 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:13.178218 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2fae35cf21340a3b90015ee4c184044', 'request_id': 'req-6465887b-b33f-42f7-a573-0190de5b75dd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:13.185421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2fae35cf21340a3b90015ee4c184044', 'request_id': 'req-6465887b-b33f-42f7-a573-0190de5b75dd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:13.191254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2fae35cf21340a3b90015ee4c184044', 'request_id': 'req-6465887b-b33f-42f7-a573-0190de5b75dd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:13.205349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] The token cache is being invalidate because project f2fae35cf21340a3b90015ee4c184044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:13.205634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:13.206308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:13.208065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 961/1926] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:13 2022] DELETE /identity/v3/projects/f2fae35cf21340a3b90015ee4c184044 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:13.214855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:13.216224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:13.224872 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fabd70-a1ca-453b-b8ed-2034d7bef4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:13.225483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fabd70-a1ca-453b-b8ed-2034d7bef4f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.225784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fabd70-a1ca-453b-b8ed-2034d7bef4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.226060 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fabd70-a1ca-453b-b8ed-2034d7bef4f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.226705 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fabd70-a1ca-453b-b8ed-2034d7bef4f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:13.228311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fabd70-a1ca-453b-b8ed-2034d7bef4f0 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:13.238131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 966/1927] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:13.241999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fabd70-a1ca-453b-b8ed-2034d7bef4f0 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:13.243639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-22fabd70-a1ca-453b-b8ed-2034d7bef4f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:13.255524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fabd70-a1ca-453b-b8ed-2034d7bef4f0 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:13.260168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc63fd0-2c4f-42c4-a4e6-468cbbd01b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:13.260743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc63fd0-2c4f-42c4-a4e6-468cbbd01b70 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.261017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc63fd0-2c4f-42c4-a4e6-468cbbd01b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.261289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc63fd0-2c4f-42c4-a4e6-468cbbd01b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.261954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc63fd0-2c4f-42c4-a4e6-468cbbd01b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:13.263527 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc63fd0-2c4f-42c4-a4e6-468cbbd01b70 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:13.274462 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 967/1928] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:13.278610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc63fd0-2c4f-42c4-a4e6-468cbbd01b70 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:13.282781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465887b-b33f-42f7-a573-0190de5b75dd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:13.287483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:13.288196 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.288539 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.288795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc63fd0-2c4f-42c4-a4e6-468cbbd01b70 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:13.288932 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] PATH_INFO: `/v3/users/bf131e3e2d3b47f09ae4cfa83fdd60f8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.290186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf131e3e2d3b47f09ae4cfa83fdd60f8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:13.292108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:13.293423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eaf5d9-992a-41ec-82a0-22e178cb2afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:13.294235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaf5d9-992a-41ec-82a0-22e178cb2afc admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.294873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaf5d9-992a-41ec-82a0-22e178cb2afc admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.295153 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaf5d9-992a-41ec-82a0-22e178cb2afc admin admin] PATH_INFO: `/v3/projects/73e9543ac314447194b3a4d851a65bf0/users/d65c95fd491f43aab82137b25aa0610d/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.309831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eaf5d9-992a-41ec-82a0-22e178cb2afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e9543ac314447194b3a4d851a65bf0, user_id=d65c95fd491f43aab82137b25aa0610d, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:13.313016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eaf5d9-992a-41ec-82a0-22e178cb2afc admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:13.328167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 968/1929] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:13 2022] PUT /identity/v3/projects/73e9543ac314447194b3a4d851a65bf0/users/d65c95fd491f43aab82137b25aa0610d/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:13.338901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8f981-53f6-4483-8062-08625a21eb19 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.339202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8f981-53f6-4483-8062-08625a21eb19 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.339464 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8f981-53f6-4483-8062-08625a21eb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.341801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf131e3e2d3b47f09ae4cfa83fdd60f8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:13.349422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf131e3e2d3b47f09ae4cfa83fdd60f8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:13.350115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf131e3e2d3b47f09ae4cfa83fdd60f8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:13.350747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf131e3e2d3b47f09ae4cfa83fdd60f8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:13.358154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf131e3e2d3b47f09ae4cfa83fdd60f8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:13.367015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-6ef8f981-53f6-4483-8062-08625a21eb19 None None] MFA Rules not processed for user `d65c95fd491f43aab82137b25aa0610d`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:13.368459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-6ef8f981-53f6-4483-8062-08625a21eb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:13.377647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 962/1930] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:13 2022] DELETE /identity/v3/users/bf131e3e2d3b47f09ae4cfa83fdd60f8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:13.438578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 969/1931] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:13.442989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:13.444494 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:13.464745 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:13.516131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a22eaa-62d1-451b-834c-f8c05b455c35 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:13.521227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76d2f8-640d-4342-810f-2d3e4bb8bb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:13.523420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76d2f8-640d-4342-810f-2d3e4bb8bb67 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.523728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76d2f8-640d-4342-810f-2d3e4bb8bb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.523990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76d2f8-640d-4342-810f-2d3e4bb8bb67 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.524645 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76d2f8-640d-4342-810f-2d3e4bb8bb67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:13.526851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76d2f8-640d-4342-810f-2d3e4bb8bb67 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:13.560680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 963/1932] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:13 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:13.562526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8f981-53f6-4483-8062-08625a21eb19 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:13.610063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:13.650138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8f981-53f6-4483-8062-08625a21eb19 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:13.722236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4a084-9abe-4a0a-b525-f512d6645bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:13.722987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4a084-9abe-4a0a-b525-f512d6645bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.723272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4a084-9abe-4a0a-b525-f512d6645bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.723549 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4a084-9abe-4a0a-b525-f512d6645bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.724194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4a084-9abe-4a0a-b525-f512d6645bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:13.726184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4a084-9abe-4a0a-b525-f512d6645bb6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:13.750001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 970/1933] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:13 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:13.753972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76d2f8-640d-4342-810f-2d3e4bb8bb67 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:13.765783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76d2f8-640d-4342-810f-2d3e4bb8bb67 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:13.771206 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef9566-906f-4b2d-8eb8-eced21b20c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:13.772069 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef9566-906f-4b2d-8eb8-eced21b20c21 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.772347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef9566-906f-4b2d-8eb8-eced21b20c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.772622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef9566-906f-4b2d-8eb8-eced21b20c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.773282 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef9566-906f-4b2d-8eb8-eced21b20c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:13.774953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef9566-906f-4b2d-8eb8-eced21b20c21 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:13.786311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 964/1934] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:13.790516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4a084-9abe-4a0a-b525-f512d6645bb6 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:13.804322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4a084-9abe-4a0a-b525-f512d6645bb6 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:13.809048 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d1f5f-93ec-4d7d-af80-d5a59103f973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:13.809848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d1f5f-93ec-4d7d-af80-d5a59103f973 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.810193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d1f5f-93ec-4d7d-af80-d5a59103f973 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.810654 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d1f5f-93ec-4d7d-af80-d5a59103f973 admin admin] PATH_INFO: `/v3/projects/04ed5108c2214df48040694ab5ad3571/users/8e3bb221e0db476280f7e49820880544/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.828003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d1f5f-93ec-4d7d-af80-d5a59103f973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ed5108c2214df48040694ab5ad3571, user_id=8e3bb221e0db476280f7e49820880544, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:13.831241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4d1f5f-93ec-4d7d-af80-d5a59103f973 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:13.848749 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 971/1935] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:13 2022] PUT /identity/v3/projects/04ed5108c2214df48040694ab5ad3571/users/8e3bb221e0db476280f7e49820880544/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:13.854000 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d1f5f-93ec-4d7d-af80-d5a59103f973 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:13.854181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef9566-906f-4b2d-8eb8-eced21b20c21 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:13.866008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-caef9566-906f-4b2d-8eb8-eced21b20c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:13.908461 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef9566-906f-4b2d-8eb8-eced21b20c21 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:13.980866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:13.981431 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:13.981715 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:13.982803 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] PATH_INFO: `/v3/projects/5d3b922f1b11426db1f5ce6a20beca57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:13.984717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d3b922f1b11426db1f5ce6a20beca57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:13.988846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:14.046792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d1f5f-93ec-4d7d-af80-d5a59103f973 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:14.056777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d3b922f1b11426db1f5ce6a20beca57', 'request_id': 'req-d21e8c8e-86d1-4ad3-9614-1d722596fd67'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:14.066437 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d3b922f1b11426db1f5ce6a20beca57', 'request_id': 'req-d21e8c8e-86d1-4ad3-9614-1d722596fd67'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:14.075822 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d3b922f1b11426db1f5ce6a20beca57', 'request_id': 'req-d21e8c8e-86d1-4ad3-9614-1d722596fd67'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:14.114851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] The token cache is being invalidate because project 5d3b922f1b11426db1f5ce6a20beca57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:14.115158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:14.131384 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:14.137599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 965/1936] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:13 2022] DELETE /identity/v3/projects/5d3b922f1b11426db1f5ce6a20beca57 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:14.142342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:14.144685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:14.212872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:14.231530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:14.253763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf148e-0808-411e-be70-badca43d8035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:14.254395 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf148e-0808-411e-be70-badca43d8035 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:14.254768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf148e-0808-411e-be70-badca43d8035 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:14.255049 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf148e-0808-411e-be70-badca43d8035 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:14.255860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf148e-0808-411e-be70-badca43d8035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:14.258043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf148e-0808-411e-be70-badca43d8035 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:14.275204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 972/1937] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:14.279297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf148e-0808-411e-be70-badca43d8035 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:14.281320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0ddf148e-0808-411e-be70-badca43d8035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:14.296464 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf148e-0808-411e-be70-badca43d8035 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:14.305586 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fb4fb-61f5-4483-aa82-d1dd2bf79f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:14.306350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fb4fb-61f5-4483-aa82-d1dd2bf79f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:14.306632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fb4fb-61f5-4483-aa82-d1dd2bf79f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:14.306907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fb4fb-61f5-4483-aa82-d1dd2bf79f86 admin admin] PATH_INFO: `/v3/projects/04ed5108c2214df48040694ab5ad3571/users/8e3bb221e0db476280f7e49820880544/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:14.309135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fb4fb-61f5-4483-aa82-d1dd2bf79f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04ed5108c2214df48040694ab5ad3571, user_id=8e3bb221e0db476280f7e49820880544, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:14.312762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fb4fb-61f5-4483-aa82-d1dd2bf79f86 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:14.326664 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 973/1938] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:14 2022] PUT /identity/v3/projects/04ed5108c2214df48040694ab5ad3571/users/8e3bb221e0db476280f7e49820880544/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:14.339763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e6bd8-f87b-4f50-9abb-30b6c2904249 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:14.340167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e6bd8-f87b-4f50-9abb-30b6c2904249 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:14.340492 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e6bd8-f87b-4f50-9abb-30b6c2904249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:14.341399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e8c8e-86d1-4ad3-9614-1d722596fd67 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:14.408177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-a48e6bd8-f87b-4f50-9abb-30b6c2904249 None None] MFA Rules not processed for user `8e3bb221e0db476280f7e49820880544`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:14.409434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a48e6bd8-f87b-4f50-9abb-30b6c2904249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:14.552086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:14.552724 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:14.553302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:14.553302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] PATH_INFO: `/v3/users/b1a1af4b9be2416aa77283fe3bbfe0b6` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:14.555189 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a1af4b9be2416aa77283fe3bbfe0b6)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:14.557157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:14.590682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 974/1939] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:14.599814 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e6bd8-f87b-4f50-9abb-30b6c2904249 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:14.603862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a48e6bd8-f87b-4f50-9abb-30b6c2904249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:14.688040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a1af4b9be2416aa77283fe3bbfe0b6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:14.705718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a1af4b9be2416aa77283fe3bbfe0b6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:14.706542 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a1af4b9be2416aa77283fe3bbfe0b6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:14.712382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a1af4b9be2416aa77283fe3bbfe0b6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:14.725617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a1af4b9be2416aa77283fe3bbfe0b6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:14.763933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e6bd8-f87b-4f50-9abb-30b6c2904249 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:14.771394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe66cdc-17f2-4add-9d80-965c270d2e73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:14.772778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe66cdc-17f2-4add-9d80-965c270d2e73 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:14.773298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe66cdc-17f2-4add-9d80-965c270d2e73 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:14.773680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe66cdc-17f2-4add-9d80-965c270d2e73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:14.777336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-cbe66cdc-17f2-4add-9d80-965c270d2e73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:14.778839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 966/1940] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:14 2022] DELETE /identity/v3/users/b1a1af4b9be2416aa77283fe3bbfe0b6 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:14.835593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe66cdc-17f2-4add-9d80-965c270d2e73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:14.841580 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe66cdc-17f2-4add-9d80-965c270d2e73 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:14.897089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 975/1941] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:15.083266 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:15.085289 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:15.199663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe66cdc-17f2-4add-9d80-965c270d2e73 service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:15.202951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-cbe66cdc-17f2-4add-9d80-965c270d2e73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:15.249592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe66cdc-17f2-4add-9d80-965c270d2e73 service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:15.253937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:15.271676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac574a4f-99fb-4d3c-a7e9-8651da3a4454 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:15.285789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab853b9-720b-45b5-bfc8-15301dc2a4ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:15.286444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab853b9-720b-45b5-bfc8-15301dc2a4ac service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.286823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab853b9-720b-45b5-bfc8-15301dc2a4ac service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.287671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab853b9-720b-45b5-bfc8-15301dc2a4ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.291023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dab853b9-720b-45b5-bfc8-15301dc2a4ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:15.339157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782a044-6e52-467f-a636-6c6bf74af04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:15.340095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782a044-6e52-467f-a636-6c6bf74af04a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.340385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782a044-6e52-467f-a636-6c6bf74af04a admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.341200 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782a044-6e52-467f-a636-6c6bf74af04a admin admin] PATH_INFO: `/v3/users/cde78890e14b4803a7ea52cddce250de` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.343428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782a044-6e52-467f-a636-6c6bf74af04a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cde78890e14b4803a7ea52cddce250de)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:15.347832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782a044-6e52-467f-a636-6c6bf74af04a admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:15.354862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:15.386195 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab853b9-720b-45b5-bfc8-15301dc2a4ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:15.387891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab853b9-720b-45b5-bfc8-15301dc2a4ac service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:15.399536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 967/1942] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:15.400828 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415b9d5-371e-450f-ad37-adcf5ea7a243 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.401171 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415b9d5-371e-450f-ad37-adcf5ea7a243 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.401385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415b9d5-371e-450f-ad37-adcf5ea7a243 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.402658 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 968/1943] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 13:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:15.403378 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415b9d5-371e-450f-ad37-adcf5ea7a243 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:15.406502 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8415b9d5-371e-450f-ad37-adcf5ea7a243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:15.409313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2782a044-6e52-467f-a636-6c6bf74af04a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cde78890e14b4803a7ea52cddce250de'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:15.419875 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2782a044-6e52-467f-a636-6c6bf74af04a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cde78890e14b4803a7ea52cddce250de'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:15.421134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2782a044-6e52-467f-a636-6c6bf74af04a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cde78890e14b4803a7ea52cddce250de'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:15.421899 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2782a044-6e52-467f-a636-6c6bf74af04a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cde78890e14b4803a7ea52cddce250de'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:15.429072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415b9d5-371e-450f-ad37-adcf5ea7a243 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:15.432137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2782a044-6e52-467f-a636-6c6bf74af04a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cde78890e14b4803a7ea52cddce250de'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:15.437585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:15.438913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.438913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.439237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b admin admin] PATH_INFO: `/v3/projects/97c64dd8363b4450b3400e94fc531483` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.440399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b admin admin] RBAC: Authorizing `identity:delete_project(project_id=97c64dd8363b4450b3400e94fc531483)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:15.442467 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:15.454656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 976/1944] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:15 2022] DELETE /identity/v3/users/cde78890e14b4803a7ea52cddce250de => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:15.456318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e0d81-e8f3-461e-97e2-8da64a8e9d98 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.456609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e0d81-e8f3-461e-97e2-8da64a8e9d98 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.456909 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e0d81-e8f3-461e-97e2-8da64a8e9d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.479554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-266e0d81-e8f3-461e-97e2-8da64a8e9d98 None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:15.480892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-266e0d81-e8f3-461e-97e2-8da64a8e9d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:15.513422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97c64dd8363b4450b3400e94fc531483', 'request_id': 'req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:15.521080 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97c64dd8363b4450b3400e94fc531483', 'request_id': 'req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:15.527616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97c64dd8363b4450b3400e94fc531483', 'request_id': 'req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:15.559022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b admin admin] The token cache is being invalidate because project 97c64dd8363b4450b3400e94fc531483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:15.559642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:15.560189 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-e9c9435c-69d3-4cf7-b1cb-b5f9849a650b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:15.564299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 969/1945] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:15 2022] DELETE /identity/v3/projects/97c64dd8363b4450b3400e94fc531483 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:15.566162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44ff05-0808-43ac-be35-c947491db231 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.566401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44ff05-0808-43ac-be35-c947491db231 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.566678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44ff05-0808-43ac-be35-c947491db231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.598201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-bc44ff05-0808-43ac-be35-c947491db231 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:15.599805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bc44ff05-0808-43ac-be35-c947491db231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:15.606366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 977/1946] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 13:53:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:15.607736 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c242279-0fee-4d1e-8469-fedd8679ec20 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.608021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c242279-0fee-4d1e-8469-fedd8679ec20 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.608393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c242279-0fee-4d1e-8469-fedd8679ec20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.630742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4c242279-0fee-4d1e-8469-fedd8679ec20 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:15.631706 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4c242279-0fee-4d1e-8469-fedd8679ec20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:15.677019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 970/1947] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:15.678688 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44ff05-0808-43ac-be35-c947491db231 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:15.680284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bc44ff05-0808-43ac-be35-c947491db231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:15.710094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 978/1948] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:15.710094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c242279-0fee-4d1e-8469-fedd8679ec20 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:15.710094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4c242279-0fee-4d1e-8469-fedd8679ec20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:15.723673 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:15.724807 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:15.786678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44ff05-0808-43ac-be35-c947491db231 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:15.791417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c242279-0fee-4d1e-8469-fedd8679ec20 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:15.792981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224860e4-0a89-45e5-8d76-b309bad44f1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:15.793920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224860e4-0a89-45e5-8d76-b309bad44f1e service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.794409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224860e4-0a89-45e5-8d76-b309bad44f1e service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.794866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224860e4-0a89-45e5-8d76-b309bad44f1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.796701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981aede7-9a2b-4aff-8c87-9fc98fd1c86b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:15.797420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981aede7-9a2b-4aff-8c87-9fc98fd1c86b service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.797864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981aede7-9a2b-4aff-8c87-9fc98fd1c86b service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.798335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981aede7-9a2b-4aff-8c87-9fc98fd1c86b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.807051 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224860e4-0a89-45e5-8d76-b309bad44f1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:15.808910 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981aede7-9a2b-4aff-8c87-9fc98fd1c86b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:15.809406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224860e4-0a89-45e5-8d76-b309bad44f1e service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:15.810776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981aede7-9a2b-4aff-8c87-9fc98fd1c86b service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:15.822190 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 971/1949] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:15.823881 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a64211-4abd-43a7-8398-23263cce1108 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.824392 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a64211-4abd-43a7-8398-23263cce1108 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.824604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 979/1950] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:15.825051 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a64211-4abd-43a7-8398-23263cce1108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.847762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e6a64211-4abd-43a7-8398-23263cce1108 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:15.848965 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e6a64211-4abd-43a7-8398-23263cce1108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:15.858740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 972/1951] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:15.864425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981aede7-9a2b-4aff-8c87-9fc98fd1c86b service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:15.885451 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981aede7-9a2b-4aff-8c87-9fc98fd1c86b service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:15.903260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52535227-f6d1-4084-a09c-ab5b2dbc6056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:15.904247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52535227-f6d1-4084-a09c-ab5b2dbc6056 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.904793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52535227-f6d1-4084-a09c-ab5b2dbc6056 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.905308 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52535227-f6d1-4084-a09c-ab5b2dbc6056 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.906222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52535227-f6d1-4084-a09c-ab5b2dbc6056 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:15.906740 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52535227-f6d1-4084-a09c-ab5b2dbc6056 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:15.910990 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52535227-f6d1-4084-a09c-ab5b2dbc6056 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:15.954783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 980/1952] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:15.957420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a64211-4abd-43a7-8398-23263cce1108 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:15.981382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a64211-4abd-43a7-8398-23263cce1108 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:15.988959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d12fb9-bfec-469e-9746-935757a0273d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:15.989595 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d12fb9-bfec-469e-9746-935757a0273d admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.989925 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d12fb9-bfec-469e-9746-935757a0273d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.990250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d12fb9-bfec-469e-9746-935757a0273d admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.990904 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d12fb9-bfec-469e-9746-935757a0273d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:16.008172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d12fb9-bfec-469e-9746-935757a0273d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:16.079005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 973/1953] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:15 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:16.079924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52535227-f6d1-4084-a09c-ab5b2dbc6056 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:16.097358 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52535227-f6d1-4084-a09c-ab5b2dbc6056 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:16.201110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6b8e1-dfd9-4eab-bf39-d65ce9001ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:16.202559 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6b8e1-dfd9-4eab-bf39-d65ce9001ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:16.202846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6b8e1-dfd9-4eab-bf39-d65ce9001ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:16.203394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6b8e1-dfd9-4eab-bf39-d65ce9001ba9 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:16.204216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6b8e1-dfd9-4eab-bf39-d65ce9001ba9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:16.207342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6b8e1-dfd9-4eab-bf39-d65ce9001ba9 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:16.246078 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 981/1954] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:16 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:16.246180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d12fb9-bfec-469e-9746-935757a0273d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:16.258084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d12fb9-bfec-469e-9746-935757a0273d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:16.263981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fdc47-45fb-48ab-aa14-f5545ac8a006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:16.265148 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fdc47-45fb-48ab-aa14-f5545ac8a006 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:16.265433 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fdc47-45fb-48ab-aa14-f5545ac8a006 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:16.265705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fdc47-45fb-48ab-aa14-f5545ac8a006 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:16.266439 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fdc47-45fb-48ab-aa14-f5545ac8a006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:16.268781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fdc47-45fb-48ab-aa14-f5545ac8a006 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:16.285320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 974/1955] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:16.286617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6b8e1-dfd9-4eab-bf39-d65ce9001ba9 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:16.298079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6b8e1-dfd9-4eab-bf39-d65ce9001ba9 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:16.303568 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf3eec-1b27-469e-9cf9-d02d945ab860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:16.304118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf3eec-1b27-469e-9cf9-d02d945ab860 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:16.304399 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf3eec-1b27-469e-9cf9-d02d945ab860 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:16.304671 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf3eec-1b27-469e-9cf9-d02d945ab860 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:16.306063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf3eec-1b27-469e-9cf9-d02d945ab860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:16.307391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf3eec-1b27-469e-9cf9-d02d945ab860 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:16.319288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 982/1956] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:16.323028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fdc47-45fb-48ab-aa14-f5545ac8a006 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:16.337756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fdc47-45fb-48ab-aa14-f5545ac8a006 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:16.344311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb399b1-235a-4a1d-bbdb-b59ee70fda3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:16.357906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb399b1-235a-4a1d-bbdb-b59ee70fda3d admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:16.358729 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb399b1-235a-4a1d-bbdb-b59ee70fda3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:16.361607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb399b1-235a-4a1d-bbdb-b59ee70fda3d admin admin] PATH_INFO: `/v3/projects/cd7812086c5947718663aca0c2da3cb1/users/4cf38c5eb57b46eabffa4a7ffa18658e/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:16.404771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb399b1-235a-4a1d-bbdb-b59ee70fda3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7812086c5947718663aca0c2da3cb1, user_id=4cf38c5eb57b46eabffa4a7ffa18658e, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:16.410091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb399b1-235a-4a1d-bbdb-b59ee70fda3d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:16.443901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 975/1957] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:16 2022] PUT /identity/v3/projects/cd7812086c5947718663aca0c2da3cb1/users/4cf38c5eb57b46eabffa4a7ffa18658e/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:16.445696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348efae4-97e1-42d8-a998-e40f44fcd333 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:16.446028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348efae4-97e1-42d8-a998-e40f44fcd333 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:16.446306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348efae4-97e1-42d8-a998-e40f44fcd333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:16.503128 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-348efae4-97e1-42d8-a998-e40f44fcd333 None None] MFA Rules not processed for user `4cf38c5eb57b46eabffa4a7ffa18658e`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:16.505914 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-348efae4-97e1-42d8-a998-e40f44fcd333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:16.587763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb399b1-235a-4a1d-bbdb-b59ee70fda3d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:16.589920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fdb399b1-235a-4a1d-bbdb-b59ee70fda3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:16.640680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 983/1958] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:16.645837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348efae4-97e1-42d8-a998-e40f44fcd333 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:16.661832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:16.739417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb399b1-235a-4a1d-bbdb-b59ee70fda3d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:16.750968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:16.751183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348efae4-97e1-42d8-a998-e40f44fcd333 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:16.752386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:16.752766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:16.753050 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] PATH_INFO: `/v3/projects/8ba6ca7d485c404dbe3ed5873c2142f6` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:16.754927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba6ca7d485c404dbe3ed5873c2142f6)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:16.758705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:16.766292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e14a91-940e-476f-bbbf-0e464c6231e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:16.767471 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e14a91-940e-476f-bbbf-0e464c6231e6 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:16.768113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e14a91-940e-476f-bbbf-0e464c6231e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:16.769381 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e14a91-940e-476f-bbbf-0e464c6231e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:16.770356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e14a91-940e-476f-bbbf-0e464c6231e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:16.772290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e14a91-940e-476f-bbbf-0e464c6231e6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:16.850618 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 984/1959] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:16 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:16.860207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e14a91-940e-476f-bbbf-0e464c6231e6 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:16.867558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:16.869341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba6ca7d485c404dbe3ed5873c2142f6', 'request_id': 'req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:16.880182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba6ca7d485c404dbe3ed5873c2142f6', 'request_id': 'req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:16.891280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba6ca7d485c404dbe3ed5873c2142f6', 'request_id': 'req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:16.920506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] The token cache is being invalidate because project 8ba6ca7d485c404dbe3ed5873c2142f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:16.920889 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:16.923053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:16.925365 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 976/1960] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:16 2022] DELETE /identity/v3/projects/8ba6ca7d485c404dbe3ed5873c2142f6 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:16.929880 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:16.932026 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:16.979343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e14a91-940e-476f-bbbf-0e464c6231e6 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:16.984336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc9e79-f66e-4f6d-9846-28046b2e16bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:16.984959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc9e79-f66e-4f6d-9846-28046b2e16bf admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:16.985383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc9e79-f66e-4f6d-9846-28046b2e16bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:16.985664 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc9e79-f66e-4f6d-9846-28046b2e16bf admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:16.986326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc9e79-f66e-4f6d-9846-28046b2e16bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:16.988272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc9e79-f66e-4f6d-9846-28046b2e16bf admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:17.017789 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 985/1961] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:16 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:17.020428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc9e79-f66e-4f6d-9846-28046b2e16bf admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:17.022652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e9fc9e79-f66e-4f6d-9846-28046b2e16bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:17.031822 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed044ddc-9bd1-4e5d-b316-5f0e49e67884 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:17.036969 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc9e79-f66e-4f6d-9846-28046b2e16bf admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:17.037599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bbcf6-3c15-4b3d-8e93-d108d5d5b8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:17.038355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bbcf6-3c15-4b3d-8e93-d108d5d5b8e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.038672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bbcf6-3c15-4b3d-8e93-d108d5d5b8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.038945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bbcf6-3c15-4b3d-8e93-d108d5d5b8e2 admin admin] PATH_INFO: `/v3/users/1a0027dadb4549fe862c63ce93823e98` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.040225 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bbcf6-3c15-4b3d-8e93-d108d5d5b8e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a0027dadb4549fe862c63ce93823e98)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:17.042249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598899c-e077-4a5a-a841-440288f596cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:17.042383 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bbcf6-3c15-4b3d-8e93-d108d5d5b8e2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:17.042865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598899c-e077-4a5a-a841-440288f596cb admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.043148 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598899c-e077-4a5a-a841-440288f596cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.043420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598899c-e077-4a5a-a841-440288f596cb admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.044296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598899c-e077-4a5a-a841-440288f596cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:17.046617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598899c-e077-4a5a-a841-440288f596cb admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:17.056933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 986/1962] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:17.061939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598899c-e077-4a5a-a841-440288f596cb admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:17.072559 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598899c-e077-4a5a-a841-440288f596cb admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:17.080338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac999609-05e0-422c-9dae-5fffe108c469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:17.080893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac999609-05e0-422c-9dae-5fffe108c469 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.081125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac999609-05e0-422c-9dae-5fffe108c469 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.081466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac999609-05e0-422c-9dae-5fffe108c469 admin admin] PATH_INFO: `/v3/projects/78973de997cf464e90a83cbee6f173a9/users/8be658cc91774e28826a6e2c06f18780/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.099293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac999609-05e0-422c-9dae-5fffe108c469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78973de997cf464e90a83cbee6f173a9, user_id=8be658cc91774e28826a6e2c06f18780, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:17.102047 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac999609-05e0-422c-9dae-5fffe108c469 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:17.105780 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d60bbcf6-3c15-4b3d-8e93-d108d5d5b8e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a0027dadb4549fe862c63ce93823e98'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.115728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d60bbcf6-3c15-4b3d-8e93-d108d5d5b8e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a0027dadb4549fe862c63ce93823e98'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.116524 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d60bbcf6-3c15-4b3d-8e93-d108d5d5b8e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a0027dadb4549fe862c63ce93823e98'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.117302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d60bbcf6-3c15-4b3d-8e93-d108d5d5b8e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a0027dadb4549fe862c63ce93823e98'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.126762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d60bbcf6-3c15-4b3d-8e93-d108d5d5b8e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a0027dadb4549fe862c63ce93823e98'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.133788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 987/1963] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:17 2022] PUT /identity/v3/projects/78973de997cf464e90a83cbee6f173a9/users/8be658cc91774e28826a6e2c06f18780/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:17.134759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac999609-05e0-422c-9dae-5fffe108c469 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:17.136575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ac999609-05e0-422c-9dae-5fffe108c469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:17.149368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 977/1964] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:16 2022] DELETE /identity/v3/users/1a0027dadb4549fe862c63ce93823e98 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:17.218411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:17.237161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac999609-05e0-422c-9dae-5fffe108c469 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:17.301691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5925d5-057e-472f-9c11-da8875080762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:17.302367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5925d5-057e-472f-9c11-da8875080762 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.302719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5925d5-057e-472f-9c11-da8875080762 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.303070 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5925d5-057e-472f-9c11-da8875080762 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.303762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5925d5-057e-472f-9c11-da8875080762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:17.305353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5925d5-057e-472f-9c11-da8875080762 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:17.325632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 988/1965] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:17.330315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bbcf6-3c15-4b3d-8e93-d108d5d5b8e2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:17.341780 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bbcf6-3c15-4b3d-8e93-d108d5d5b8e2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:17.348544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d4b5f-d19a-4a49-8f8a-73998f12cddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:17.349233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d4b5f-d19a-4a49-8f8a-73998f12cddb admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.349514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d4b5f-d19a-4a49-8f8a-73998f12cddb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.350105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d4b5f-d19a-4a49-8f8a-73998f12cddb admin admin] PATH_INFO: `/v3/projects/78973de997cf464e90a83cbee6f173a9/users/8be658cc91774e28826a6e2c06f18780/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.353024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d4b5f-d19a-4a49-8f8a-73998f12cddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=78973de997cf464e90a83cbee6f173a9, user_id=8be658cc91774e28826a6e2c06f18780, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:17.356153 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d4b5f-d19a-4a49-8f8a-73998f12cddb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:17.373426 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 978/1966] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:17 2022] PUT /identity/v3/projects/78973de997cf464e90a83cbee6f173a9/users/8be658cc91774e28826a6e2c06f18780/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:17.383066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315337f8-da27-48bd-9947-c047e6f3f0ca None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.383353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315337f8-da27-48bd-9947-c047e6f3f0ca None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.383653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315337f8-da27-48bd-9947-c047e6f3f0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.418980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-315337f8-da27-48bd-9947-c047e6f3f0ca None None] MFA Rules not processed for user `8be658cc91774e28826a6e2c06f18780`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:17.420830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-315337f8-da27-48bd-9947-c047e6f3f0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:17.474635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d4b5f-d19a-4a49-8f8a-73998f12cddb admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:17.477095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e50d4b5f-d19a-4a49-8f8a-73998f12cddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:17.522281 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 989/1967] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:17.523142 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315337f8-da27-48bd-9947-c047e6f3f0ca None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:17.524436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-315337f8-da27-48bd-9947-c047e6f3f0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:17.543677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:17.566637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d4b5f-d19a-4a49-8f8a-73998f12cddb admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:17.572017 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:17.572647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.572925 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.573200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] PATH_INFO: `/v3/projects/f48ad170248c4705b811cd802616e460` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.574427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48ad170248c4705b811cd802616e460)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:17.576389 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:17.593872 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315337f8-da27-48bd-9947-c047e6f3f0ca None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:17.598879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6b013-2243-442d-8ab0-66dfee35d2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:17.599521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6b013-2243-442d-8ab0-66dfee35d2d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.599799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6b013-2243-442d-8ab0-66dfee35d2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.600137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6b013-2243-442d-8ab0-66dfee35d2d4 admin admin] PATH_INFO: `/v3/users/8406fab60cb4464996b8d929da311f89` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.601343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6b013-2243-442d-8ab0-66dfee35d2d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8406fab60cb4464996b8d929da311f89)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:17.603304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc6b013-2243-442d-8ab0-66dfee35d2d4 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:17.630917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48ad170248c4705b811cd802616e460', 'request_id': 'req-44d7f111-46cc-4401-b70b-053aabff1f39'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.632141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:17.639836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48ad170248c4705b811cd802616e460', 'request_id': 'req-44d7f111-46cc-4401-b70b-053aabff1f39'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.647935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48ad170248c4705b811cd802616e460', 'request_id': 'req-44d7f111-46cc-4401-b70b-053aabff1f39'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.662793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] The token cache is being invalidate because project f48ad170248c4705b811cd802616e460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:17.663131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.663856 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.670207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 979/1968] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:17 2022] DELETE /identity/v3/projects/f48ad170248c4705b811cd802616e460 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:17.672201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:17.674216 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:17.678837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-adc6b013-2243-442d-8ab0-66dfee35d2d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8406fab60cb4464996b8d929da311f89'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.690498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-adc6b013-2243-442d-8ab0-66dfee35d2d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8406fab60cb4464996b8d929da311f89'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.692176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-adc6b013-2243-442d-8ab0-66dfee35d2d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8406fab60cb4464996b8d929da311f89'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.693132 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-adc6b013-2243-442d-8ab0-66dfee35d2d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8406fab60cb4464996b8d929da311f89'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.701573 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-adc6b013-2243-442d-8ab0-66dfee35d2d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8406fab60cb4464996b8d929da311f89'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:17.724566 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 990/1969] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:17 2022] DELETE /identity/v3/users/8406fab60cb4464996b8d929da311f89 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:17.727154 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9437d-b58c-4d52-b346-b85981c5a7fd None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.727509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9437d-b58c-4d52-b346-b85981c5a7fd None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.728087 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d9437d-b58c-4d52-b346-b85981c5a7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.748841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-36d9437d-b58c-4d52-b346-b85981c5a7fd None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:17.750092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-36d9437d-b58c-4d52-b346-b85981c5a7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:17.753055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7f111-46cc-4401-b70b-053aabff1f39 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:17.809799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad214d2-8403-4d23-b3e3-d0ec0dc734bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:17.810452 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad214d2-8403-4d23-b3e3-d0ec0dc734bd service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.810732 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad214d2-8403-4d23-b3e3-d0ec0dc734bd service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.811008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad214d2-8403-4d23-b3e3-d0ec0dc734bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.813683 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3ad214d2-8403-4d23-b3e3-d0ec0dc734bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:17.819270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 991/1970] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:17.820727 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2b68f-4246-41f1-ac29-a226e852d47c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.821091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2b68f-4246-41f1-ac29-a226e852d47c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.821439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2b68f-4246-41f1-ac29-a226e852d47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.842135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-f5a2b68f-4246-41f1-ac29-a226e852d47c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:17.843189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f5a2b68f-4246-41f1-ac29-a226e852d47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:17.852155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 992/1971] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:17.853498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36751251-930c-4017-b38f-49fb1ba5f514 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.853821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36751251-930c-4017-b38f-49fb1ba5f514 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.854099 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36751251-930c-4017-b38f-49fb1ba5f514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.873550 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:17.877913 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-36751251-930c-4017-b38f-49fb1ba5f514 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:17.879143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-36751251-930c-4017-b38f-49fb1ba5f514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:17.888021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 993/1972] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:17.888943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36751251-930c-4017-b38f-49fb1ba5f514 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:17.893153 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-36751251-930c-4017-b38f-49fb1ba5f514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:17.924011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad214d2-8403-4d23-b3e3-d0ec0dc734bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:17.925593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad214d2-8403-4d23-b3e3-d0ec0dc734bd service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:17.940706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 980/1973] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:17.941772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad214d2-8403-4d23-b3e3-d0ec0dc734bd service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:17.962424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad214d2-8403-4d23-b3e3-d0ec0dc734bd service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:17.970047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c3367-4065-4ee1-b212-62cf9a6589fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:17.970671 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c3367-4065-4ee1-b212-62cf9a6589fb admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.970950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c3367-4065-4ee1-b212-62cf9a6589fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.972096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c3367-4065-4ee1-b212-62cf9a6589fb admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:17.972096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c3367-4065-4ee1-b212-62cf9a6589fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:17.972290 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c3367-4065-4ee1-b212-62cf9a6589fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:17.973927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765c3367-4065-4ee1-b212-62cf9a6589fb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:17.981608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36751251-930c-4017-b38f-49fb1ba5f514 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:17.985870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf99cf-12a8-4aa2-bec4-daba115869c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:17.986293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 981/1974] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:17.990398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf99cf-12a8-4aa2-bec4-daba115869c6 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:17.990626 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c3367-4065-4ee1-b212-62cf9a6589fb admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:17.991174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf99cf-12a8-4aa2-bec4-daba115869c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:17.991472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf99cf-12a8-4aa2-bec4-daba115869c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.005677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c3367-4065-4ee1-b212-62cf9a6589fb admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:18.011113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cefdbe-f511-4ccb-ba85-02d31c223a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:18.011869 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cefdbe-f511-4ccb-ba85-02d31c223a55 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:18.012150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cefdbe-f511-4ccb-ba85-02d31c223a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:18.012424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cefdbe-f511-4ccb-ba85-02d31c223a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.013064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cefdbe-f511-4ccb-ba85-02d31c223a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:18.015745 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cefdbe-f511-4ccb-ba85-02d31c223a55 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:18.023954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf99cf-12a8-4aa2-bec4-daba115869c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:18.026501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf99cf-12a8-4aa2-bec4-daba115869c6 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:18.046690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 994/1975] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:18.064725 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 982/1976] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:17 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:18.066840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf99cf-12a8-4aa2-bec4-daba115869c6 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:18.066964 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cefdbe-f511-4ccb-ba85-02d31c223a55 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:18.068411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-86cf99cf-12a8-4aa2-bec4-daba115869c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:18.083227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf99cf-12a8-4aa2-bec4-daba115869c6 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:18.128093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:18.166558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cefdbe-f511-4ccb-ba85-02d31c223a55 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:18.170901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae702a32-7a10-465c-942f-3ef4b4e6f20d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:18.172071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae702a32-7a10-465c-942f-3ef4b4e6f20d service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:18.172071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae702a32-7a10-465c-942f-3ef4b4e6f20d service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:18.172309 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fcf53-d16a-4904-8bab-6270ee24f2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:18.172934 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fcf53-d16a-4904-8bab-6270ee24f2ed admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:18.173221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fcf53-d16a-4904-8bab-6270ee24f2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:18.173904 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fcf53-d16a-4904-8bab-6270ee24f2ed admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.174078 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae702a32-7a10-465c-942f-3ef4b4e6f20d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.174846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827fcf53-d16a-4904-8bab-6270ee24f2ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:18.176532 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ae702a32-7a10-465c-942f-3ef4b4e6f20d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:18.177687 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827fcf53-d16a-4904-8bab-6270ee24f2ed admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:18.214916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 983/1977] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:18 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:18.216089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fcf53-d16a-4904-8bab-6270ee24f2ed admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:18.229706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fcf53-d16a-4904-8bab-6270ee24f2ed admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:18.236359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44755341-8fba-452d-9ee7-7048c0ac760f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:18.238182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44755341-8fba-452d-9ee7-7048c0ac760f admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:18.238182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44755341-8fba-452d-9ee7-7048c0ac760f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:18.238182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44755341-8fba-452d-9ee7-7048c0ac760f admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.238571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44755341-8fba-452d-9ee7-7048c0ac760f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:18.241232 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44755341-8fba-452d-9ee7-7048c0ac760f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:18.263420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 984/1978] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:18.264673 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44755341-8fba-452d-9ee7-7048c0ac760f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:18.266221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae702a32-7a10-465c-942f-3ef4b4e6f20d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:18.268401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae702a32-7a10-465c-942f-3ef4b4e6f20d service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:18.281237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 995/1979] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:18.289942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44755341-8fba-452d-9ee7-7048c0ac760f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:18.296940 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe219dfc-f60b-42a4-8bd5-6818e38965fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:18.298089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe219dfc-f60b-42a4-8bd5-6818e38965fa admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:18.298375 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe219dfc-f60b-42a4-8bd5-6818e38965fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:18.298653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe219dfc-f60b-42a4-8bd5-6818e38965fa admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.299897 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe219dfc-f60b-42a4-8bd5-6818e38965fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:18.301969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe219dfc-f60b-42a4-8bd5-6818e38965fa admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:18.318381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 985/1980] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:18.324328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae702a32-7a10-465c-942f-3ef4b4e6f20d service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:18.335126 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae702a32-7a10-465c-942f-3ef4b4e6f20d service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:18.340372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f47112-66c4-4799-b40c-e285fa8caec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:18.340938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f47112-66c4-4799-b40c-e285fa8caec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:18.341215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f47112-66c4-4799-b40c-e285fa8caec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:18.341489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f47112-66c4-4799-b40c-e285fa8caec2 admin admin] PATH_INFO: `/v3/projects/a38673eb4b9348e1b768f7e568e1fdcf/users/dea1ba3cb43a45c4884ef55746930959/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.378980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f47112-66c4-4799-b40c-e285fa8caec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38673eb4b9348e1b768f7e568e1fdcf, user_id=dea1ba3cb43a45c4884ef55746930959, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:18.383636 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f47112-66c4-4799-b40c-e285fa8caec2 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:18.411617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 996/1981] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:18 2022] PUT /identity/v3/projects/a38673eb4b9348e1b768f7e568e1fdcf/users/dea1ba3cb43a45c4884ef55746930959/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:18.424090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c3ed1-dda5-42da-89e2-239897faa546 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:18.425107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c3ed1-dda5-42da-89e2-239897faa546 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:18.425540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c3ed1-dda5-42da-89e2-239897faa546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.468688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f47112-66c4-4799-b40c-e285fa8caec2 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:18.471329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a3f47112-66c4-4799-b40c-e285fa8caec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:18.482827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-723c3ed1-dda5-42da-89e2-239897faa546 None None] MFA Rules not processed for user `dea1ba3cb43a45c4884ef55746930959`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:18.484418 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-723c3ed1-dda5-42da-89e2-239897faa546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:18.600475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f47112-66c4-4799-b40c-e285fa8caec2 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:18.607028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:18.607685 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:18.609133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:18.609445 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] PATH_INFO: `/v3/projects/fc5175fe08c240a9b090258cf832294c` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.612056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:18.613831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc5175fe08c240a9b090258cf832294c)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:18.622245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:18.624917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 986/1982] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:18.625790 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c3ed1-dda5-42da-89e2-239897faa546 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:18.646336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c3ed1-dda5-42da-89e2-239897faa546 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:18.651459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2ccb7-4d42-4276-b0cc-b212841c472f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:18.652195 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2ccb7-4d42-4276-b0cc-b212841c472f admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:18.652538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2ccb7-4d42-4276-b0cc-b212841c472f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:18.652879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2ccb7-4d42-4276-b0cc-b212841c472f admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.653596 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2ccb7-4d42-4276-b0cc-b212841c472f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:18.656896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2ccb7-4d42-4276-b0cc-b212841c472f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:18.706742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 987/1983] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:18 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:18.710601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2ccb7-4d42-4276-b0cc-b212841c472f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:18.718052 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc5175fe08c240a9b090258cf832294c', 'request_id': 'req-1ad86040-ee83-45ff-a2a1-30650487f753'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:18.731664 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc5175fe08c240a9b090258cf832294c', 'request_id': 'req-1ad86040-ee83-45ff-a2a1-30650487f753'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:18.744664 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc5175fe08c240a9b090258cf832294c', 'request_id': 'req-1ad86040-ee83-45ff-a2a1-30650487f753'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:18.747233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:18.765256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] The token cache is being invalidate because project fc5175fe08c240a9b090258cf832294c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:18.766549 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:18.767446 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:18.769909 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 997/1984] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:18 2022] DELETE /identity/v3/projects/fc5175fe08c240a9b090258cf832294c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:18.770719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:18.773904 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:18.801987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2ccb7-4d42-4276-b0cc-b212841c472f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:18.809509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827af93-0619-4d74-905f-31da365a7575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:18.812402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827af93-0619-4d74-905f-31da365a7575 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:18.813223 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827af93-0619-4d74-905f-31da365a7575 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:18.813663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827af93-0619-4d74-905f-31da365a7575 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.814535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827af93-0619-4d74-905f-31da365a7575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:18.818549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827af93-0619-4d74-905f-31da365a7575 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:18.888002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad86040-ee83-45ff-a2a1-30650487f753 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:18.889144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 988/1985] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:18 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:18.890149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827af93-0619-4d74-905f-31da365a7575 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:18.892116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5827af93-0619-4d74-905f-31da365a7575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:18.895149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:18.896231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:18.896759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:18.897429 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] PATH_INFO: `/v3/users/4cf38c5eb57b46eabffa4a7ffa18658e` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.899726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf38c5eb57b46eabffa4a7ffa18658e)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:18.902848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:18.924674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827af93-0619-4d74-905f-31da365a7575 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:18.939585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a7050-8784-438b-a2a7-d73ff7477377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:18.942406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a7050-8784-438b-a2a7-d73ff7477377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:18.943308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a7050-8784-438b-a2a7-d73ff7477377 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:18.944339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a7050-8784-438b-a2a7-d73ff7477377 admin admin] PATH_INFO: `/v3/users/48e0fa0c29d845bc850b8c9400fc8f7f` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:18.949094 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a7050-8784-438b-a2a7-d73ff7477377 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48e0fa0c29d845bc850b8c9400fc8f7f)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:18.957904 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a7050-8784-438b-a2a7-d73ff7477377 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:19.016788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf38c5eb57b46eabffa4a7ffa18658e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:19.036690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf38c5eb57b46eabffa4a7ffa18658e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:19.038109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf38c5eb57b46eabffa4a7ffa18658e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:19.039039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf38c5eb57b46eabffa4a7ffa18658e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:19.058131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf38c5eb57b46eabffa4a7ffa18658e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:19.099411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 998/1986] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:18 2022] DELETE /identity/v3/users/4cf38c5eb57b46eabffa4a7ffa18658e => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:19.101347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:19.102890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:19.106040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-657a7050-8784-438b-a2a7-d73ff7477377 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48e0fa0c29d845bc850b8c9400fc8f7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:19.116153 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-657a7050-8784-438b-a2a7-d73ff7477377 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48e0fa0c29d845bc850b8c9400fc8f7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:19.117101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-657a7050-8784-438b-a2a7-d73ff7477377 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48e0fa0c29d845bc850b8c9400fc8f7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:19.117896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-657a7050-8784-438b-a2a7-d73ff7477377 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48e0fa0c29d845bc850b8c9400fc8f7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:19.128813 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-657a7050-8784-438b-a2a7-d73ff7477377 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48e0fa0c29d845bc850b8c9400fc8f7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:19.171444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 989/1987] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:18 2022] DELETE /identity/v3/users/48e0fa0c29d845bc850b8c9400fc8f7f => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:19.171444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f506fa-e9ad-4857-af1d-e6bfdc8df656 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:19.171444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f506fa-e9ad-4857-af1d-e6bfdc8df656 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:19.171444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f506fa-e9ad-4857-af1d-e6bfdc8df656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:19.193876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-76f506fa-e9ad-4857-af1d-e6bfdc8df656 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:19.195195 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-76f506fa-e9ad-4857-af1d-e6bfdc8df656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:19.222630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab22cd8-7473-4486-8077-62228d470d35 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:19.249846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:19.267907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:19.287494 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 990/1988] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:19.383748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f506fa-e9ad-4857-af1d-e6bfdc8df656 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:19.385607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-76f506fa-e9ad-4857-af1d-e6bfdc8df656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:19.409589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee6fc6-8452-4291-a3ca-77fdd85e0dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:19.409589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee6fc6-8452-4291-a3ca-77fdd85e0dbd admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:19.409589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee6fc6-8452-4291-a3ca-77fdd85e0dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:19.409589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee6fc6-8452-4291-a3ca-77fdd85e0dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:19.410476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee6fc6-8452-4291-a3ca-77fdd85e0dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:19.410679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee6fc6-8452-4291-a3ca-77fdd85e0dbd admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:19.444426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 999/1989] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:19.450853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee6fc6-8452-4291-a3ca-77fdd85e0dbd admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:19.453400 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-46ee6fc6-8452-4291-a3ca-77fdd85e0dbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:19.475825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee6fc6-8452-4291-a3ca-77fdd85e0dbd admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:19.484002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375aba77-957d-4e86-8bce-a7ec41ccef6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:19.485069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375aba77-957d-4e86-8bce-a7ec41ccef6b admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:19.485563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375aba77-957d-4e86-8bce-a7ec41ccef6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:19.486097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375aba77-957d-4e86-8bce-a7ec41ccef6b admin admin] PATH_INFO: `/v3/projects/e37118c23ace494ebb71802e04e4b4e1/users/0a5d02feb8724947913fdbb1742d005b/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:19.515260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375aba77-957d-4e86-8bce-a7ec41ccef6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e37118c23ace494ebb71802e04e4b4e1, user_id=0a5d02feb8724947913fdbb1742d005b, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:19.517942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f506fa-e9ad-4857-af1d-e6bfdc8df656 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:19.520122 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375aba77-957d-4e86-8bce-a7ec41ccef6b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:19.521950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3375b-840e-495f-829a-a07077a99c4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:19.522683 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3375b-840e-495f-829a-a07077a99c4f service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:19.523068 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3375b-840e-495f-829a-a07077a99c4f service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:19.523448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3375b-840e-495f-829a-a07077a99c4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:19.536060 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3375b-840e-495f-829a-a07077a99c4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:19.537584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3375b-840e-495f-829a-a07077a99c4f service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:19.541513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1000/1990] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:19 2022] PUT /identity/v3/projects/e37118c23ace494ebb71802e04e4b4e1/users/0a5d02feb8724947913fdbb1742d005b/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:19.545043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375aba77-957d-4e86-8bce-a7ec41ccef6b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:19.550835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 991/1991] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:19.623605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375aba77-957d-4e86-8bce-a7ec41ccef6b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:19.629175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c214bda-a843-4817-ae7a-54df55a54511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:19.629986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c214bda-a843-4817-ae7a-54df55a54511 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:19.630341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c214bda-a843-4817-ae7a-54df55a54511 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:19.630620 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c214bda-a843-4817-ae7a-54df55a54511 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:19.631466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c214bda-a843-4817-ae7a-54df55a54511 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:19.633291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c214bda-a843-4817-ae7a-54df55a54511 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:19.647542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1001/1992] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:19.651700 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3375b-840e-495f-829a-a07077a99c4f service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:19.662703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3375b-840e-495f-829a-a07077a99c4f service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:19.667361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d7b0b-bec1-4d94-9a0d-d7f4fba0925a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:19.668202 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d7b0b-bec1-4d94-9a0d-d7f4fba0925a admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:19.668339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d7b0b-bec1-4d94-9a0d-d7f4fba0925a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:19.668576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d7b0b-bec1-4d94-9a0d-d7f4fba0925a admin admin] PATH_INFO: `/v3/projects/e37118c23ace494ebb71802e04e4b4e1/users/0a5d02feb8724947913fdbb1742d005b/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:19.671277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d7b0b-bec1-4d94-9a0d-d7f4fba0925a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e37118c23ace494ebb71802e04e4b4e1, user_id=0a5d02feb8724947913fdbb1742d005b, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:19.674025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d7b0b-bec1-4d94-9a0d-d7f4fba0925a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:19.685884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 992/1993] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:19 2022] PUT /identity/v3/projects/e37118c23ace494ebb71802e04e4b4e1/users/0a5d02feb8724947913fdbb1742d005b/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:19.696152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0eb1e-06d7-4111-8445-456f415a6de9 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:19.696454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0eb1e-06d7-4111-8445-456f415a6de9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:19.696710 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0eb1e-06d7-4111-8445-456f415a6de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:19.730936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-77c0eb1e-06d7-4111-8445-456f415a6de9 None None] MFA Rules not processed for user `0a5d02feb8724947913fdbb1742d005b`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:19.732441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-77c0eb1e-06d7-4111-8445-456f415a6de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:19.820906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1002/1994] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:19.833775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d7b0b-bec1-4d94-9a0d-d7f4fba0925a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:19.834758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-126d7b0b-bec1-4d94-9a0d-d7f4fba0925a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:19.847461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0eb1e-06d7-4111-8445-456f415a6de9 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:19.848991 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-77c0eb1e-06d7-4111-8445-456f415a6de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:19.861404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:19.921452 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d7b0b-bec1-4d94-9a0d-d7f4fba0925a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:19.925775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397ae91-830a-402e-b293-8e39cca437e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:19.926615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397ae91-830a-402e-b293-8e39cca437e5 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:19.926758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397ae91-830a-402e-b293-8e39cca437e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:19.927122 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b397ae91-830a-402e-b293-8e39cca437e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:19.945763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397ae91-830a-402e-b293-8e39cca437e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:19.947453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b397ae91-830a-402e-b293-8e39cca437e5 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:19.949806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0eb1e-06d7-4111-8445-456f415a6de9 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:19.956229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f719ca7-1f8e-4123-8275-4cd784ce6876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:19.957044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f719ca7-1f8e-4123-8275-4cd784ce6876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:19.957689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f719ca7-1f8e-4123-8275-4cd784ce6876 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:19.958143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f719ca7-1f8e-4123-8275-4cd784ce6876 admin admin] PATH_INFO: `/v3/projects/af8a9ff7e3bd4df1bec68a1fc346a311` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:19.958889 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 993/1995] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:19.962629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f719ca7-1f8e-4123-8275-4cd784ce6876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af8a9ff7e3bd4df1bec68a1fc346a311)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:19.964856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f719ca7-1f8e-4123-8275-4cd784ce6876 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:20.001232 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397ae91-830a-402e-b293-8e39cca437e5 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:20.015744 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b397ae91-830a-402e-b293-8e39cca437e5 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:20.039023 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-6f719ca7-1f8e-4123-8275-4cd784ce6876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af8a9ff7e3bd4df1bec68a1fc346a311', 'request_id': 'req-6f719ca7-1f8e-4123-8275-4cd784ce6876'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.046932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-6f719ca7-1f8e-4123-8275-4cd784ce6876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af8a9ff7e3bd4df1bec68a1fc346a311', 'request_id': 'req-6f719ca7-1f8e-4123-8275-4cd784ce6876'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.056178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-6f719ca7-1f8e-4123-8275-4cd784ce6876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af8a9ff7e3bd4df1bec68a1fc346a311', 'request_id': 'req-6f719ca7-1f8e-4123-8275-4cd784ce6876'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.070993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-6f719ca7-1f8e-4123-8275-4cd784ce6876 admin admin] The token cache is being invalidate because project af8a9ff7e3bd4df1bec68a1fc346a311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:20.071340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-6f719ca7-1f8e-4123-8275-4cd784ce6876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.072289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-6f719ca7-1f8e-4123-8275-4cd784ce6876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.075028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1003/1996] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:19 2022] DELETE /identity/v3/projects/af8a9ff7e3bd4df1bec68a1fc346a311 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:20.080157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c03196-344d-473f-ba06-5a6362b5a8d9 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.080457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c03196-344d-473f-ba06-5a6362b5a8d9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.080728 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c03196-344d-473f-ba06-5a6362b5a8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.104207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052eb6bd-6db6-41a5-95c1-035da037089c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:20.104543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-66c03196-344d-473f-ba06-5a6362b5a8d9 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:20.105023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052eb6bd-6db6-41a5-95c1-035da037089c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.105308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052eb6bd-6db6-41a5-95c1-035da037089c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.105580 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052eb6bd-6db6-41a5-95c1-035da037089c admin admin] PATH_INFO: `/v3/users/dea1ba3cb43a45c4884ef55746930959` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.105799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-66c03196-344d-473f-ba06-5a6362b5a8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.106917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052eb6bd-6db6-41a5-95c1-035da037089c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dea1ba3cb43a45c4884ef55746930959)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:20.108956 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052eb6bd-6db6-41a5-95c1-035da037089c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:20.121624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1004/1997] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:20.125109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8bd15-6d4e-4fd2-83ae-77f6bfad0618 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.125402 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8bd15-6d4e-4fd2-83ae-77f6bfad0618 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.125685 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8bd15-6d4e-4fd2-83ae-77f6bfad0618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.126203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:20.151661 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b1b8bd15-6d4e-4fd2-83ae-77f6bfad0618 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:20.152928 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b1b8bd15-6d4e-4fd2-83ae-77f6bfad0618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.161342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1005/1998] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:20.165614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8bd15-6d4e-4fd2-83ae-77f6bfad0618 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:20.165901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-052eb6bd-6db6-41a5-95c1-035da037089c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dea1ba3cb43a45c4884ef55746930959'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.174677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-052eb6bd-6db6-41a5-95c1-035da037089c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dea1ba3cb43a45c4884ef55746930959'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.175335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-052eb6bd-6db6-41a5-95c1-035da037089c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dea1ba3cb43a45c4884ef55746930959'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.175512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8bd15-6d4e-4fd2-83ae-77f6bfad0618 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:20.176200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-052eb6bd-6db6-41a5-95c1-035da037089c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dea1ba3cb43a45c4884ef55746930959'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.180742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbef18a-0302-46f0-b682-7ef11b097547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:20.181357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbef18a-0302-46f0-b682-7ef11b097547 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.181638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbef18a-0302-46f0-b682-7ef11b097547 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.182081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbef18a-0302-46f0-b682-7ef11b097547 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.182736 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbef18a-0302-46f0-b682-7ef11b097547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:20.183416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbef18a-0302-46f0-b682-7ef11b097547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:20.183855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-052eb6bd-6db6-41a5-95c1-035da037089c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dea1ba3cb43a45c4884ef55746930959'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.184982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbef18a-0302-46f0-b682-7ef11b097547 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:20.199189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1006/1999] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:53:20.204781 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbef18a-0302-46f0-b682-7ef11b097547 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:20.206588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-cfbef18a-0302-46f0-b682-7ef11b097547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.206709 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 994/2000] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:20 2022] DELETE /identity/v3/users/dea1ba3cb43a45c4884ef55746930959 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:20.211955 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d7649-8cf1-4ba4-a9ed-ce9eecf8dbe4 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.212445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d7649-8cf1-4ba4-a9ed-ce9eecf8dbe4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.212879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d7649-8cf1-4ba4-a9ed-ce9eecf8dbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.223220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbef18a-0302-46f0-b682-7ef11b097547 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:20.239060 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-433d7649-8cf1-4ba4-a9ed-ce9eecf8dbe4 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:20.240297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-433d7649-8cf1-4ba4-a9ed-ce9eecf8dbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.293994 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e7d83-bf94-4090-a33f-e423d2b1c106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:20.295112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e7d83-bf94-4090-a33f-e423d2b1c106 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.295396 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e7d83-bf94-4090-a33f-e423d2b1c106 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.295695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e7d83-bf94-4090-a33f-e423d2b1c106 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.296328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e7d83-bf94-4090-a33f-e423d2b1c106 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:20.298530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e7d83-bf94-4090-a33f-e423d2b1c106 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:20.308352 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 995/2001] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:20.311097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d7649-8cf1-4ba4-a9ed-ce9eecf8dbe4 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:20.313267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-433d7649-8cf1-4ba4-a9ed-ce9eecf8dbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.321800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:20.326419 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d7649-8cf1-4ba4-a9ed-ce9eecf8dbe4 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:20.332342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:20.332995 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.333278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.333575 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] PATH_INFO: `/v3/projects/cd7812086c5947718663aca0c2da3cb1` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.334403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1007/2002] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:20 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:20.335243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7812086c5947718663aca0c2da3cb1)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:20.335759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e7d83-bf94-4090-a33f-e423d2b1c106 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:20.338083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:20.338376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ff1e7d83-bf94-4090-a33f-e423d2b1c106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.392120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7812086c5947718663aca0c2da3cb1', 'request_id': 'req-03e4cb66-d18d-46cb-8893-898d36305bb4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.399327 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7812086c5947718663aca0c2da3cb1', 'request_id': 'req-03e4cb66-d18d-46cb-8893-898d36305bb4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.401219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e7d83-bf94-4090-a33f-e423d2b1c106 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:20.405394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7812086c5947718663aca0c2da3cb1', 'request_id': 'req-03e4cb66-d18d-46cb-8893-898d36305bb4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.419323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] The token cache is being invalidate because project cd7812086c5947718663aca0c2da3cb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:20.419614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.420347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.423025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 996/2003] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:20 2022] DELETE /identity/v3/projects/cd7812086c5947718663aca0c2da3cb1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:20.423612 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:20.425008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.447033 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:20.481067 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952650d-14a1-440d-92d1-0a0f0d9ba6c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:20.481710 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952650d-14a1-440d-92d1-0a0f0d9ba6c4 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.482024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952650d-14a1-440d-92d1-0a0f0d9ba6c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.482475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952650d-14a1-440d-92d1-0a0f0d9ba6c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.485559 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5952650d-14a1-440d-92d1-0a0f0d9ba6c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.524413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4cb66-d18d-46cb-8893-898d36305bb4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:20.530971 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba377d27-6e04-4810-9927-c050edabb9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:20.531969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba377d27-6e04-4810-9927-c050edabb9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.532504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba377d27-6e04-4810-9927-c050edabb9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.532946 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba377d27-6e04-4810-9927-c050edabb9d7 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.533793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba377d27-6e04-4810-9927-c050edabb9d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:20.535906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba377d27-6e04-4810-9927-c050edabb9d7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:20.565065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 997/2004] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:20 2022] POST /identity/v3/users => generated 286 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:20.566182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba377d27-6e04-4810-9927-c050edabb9d7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:20.567901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ba377d27-6e04-4810-9927-c050edabb9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.575340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952650d-14a1-440d-92d1-0a0f0d9ba6c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:20.576795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5952650d-14a1-440d-92d1-0a0f0d9ba6c4 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:20.581388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba377d27-6e04-4810-9927-c050edabb9d7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:20.587540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1008/2005] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:20.588304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952650d-14a1-440d-92d1-0a0f0d9ba6c4 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:20.589378 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d06b0-cfc2-4898-a671-6414efc98299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:20.590126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d06b0-cfc2-4898-a671-6414efc98299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.590482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d06b0-cfc2-4898-a671-6414efc98299 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.591184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d06b0-cfc2-4898-a671-6414efc98299 admin admin] PATH_INFO: `/v3/users/8be658cc91774e28826a6e2c06f18780` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.592673 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d06b0-cfc2-4898-a671-6414efc98299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8be658cc91774e28826a6e2c06f18780)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:20.594873 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d06b0-cfc2-4898-a671-6414efc98299 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:20.598526 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952650d-14a1-440d-92d1-0a0f0d9ba6c4 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:20.610040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c374e26-1883-484c-9bb7-01272034a407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:20.610666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374e26-1883-484c-9bb7-01272034a407 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.610944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374e26-1883-484c-9bb7-01272034a407 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.611212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374e26-1883-484c-9bb7-01272034a407 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.611897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c374e26-1883-484c-9bb7-01272034a407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:20.613799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c374e26-1883-484c-9bb7-01272034a407 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:20.625204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1009/2006] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:20.629381 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c374e26-1883-484c-9bb7-01272034a407 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:20.639990 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c374e26-1883-484c-9bb7-01272034a407 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:20.645681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111e613-8ee6-4fdf-a503-abca6f4507c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:20.646309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111e613-8ee6-4fdf-a503-abca6f4507c4 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.646585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111e613-8ee6-4fdf-a503-abca6f4507c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.646846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111e613-8ee6-4fdf-a503-abca6f4507c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.647524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111e613-8ee6-4fdf-a503-abca6f4507c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:20.649119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111e613-8ee6-4fdf-a503-abca6f4507c4 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:20.653580 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b55d06b0-cfc2-4898-a671-6414efc98299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8be658cc91774e28826a6e2c06f18780'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.660464 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1010/2007] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:20.662945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b55d06b0-cfc2-4898-a671-6414efc98299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8be658cc91774e28826a6e2c06f18780'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.664160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b55d06b0-cfc2-4898-a671-6414efc98299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8be658cc91774e28826a6e2c06f18780'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.664956 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b55d06b0-cfc2-4898-a671-6414efc98299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8be658cc91774e28826a6e2c06f18780'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.665301 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111e613-8ee6-4fdf-a503-abca6f4507c4 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:20.667523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-9111e613-8ee6-4fdf-a503-abca6f4507c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.674537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b55d06b0-cfc2-4898-a671-6414efc98299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8be658cc91774e28826a6e2c06f18780'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:20.683090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111e613-8ee6-4fdf-a503-abca6f4507c4 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:20.690905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4627ac-88ac-44b8-81b8-de21c269c5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:20.691870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4627ac-88ac-44b8-81b8-de21c269c5e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.692229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4627ac-88ac-44b8-81b8-de21c269c5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.692519 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4627ac-88ac-44b8-81b8-de21c269c5e7 admin admin] PATH_INFO: `/v3/projects/ddf1cad193fa4fd184835f2ac08ae25a/users/884441a80e9f4d26a4fc04be59478120/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.707022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 998/2008] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:20 2022] DELETE /identity/v3/users/8be658cc91774e28826a6e2c06f18780 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:20.718213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4627ac-88ac-44b8-81b8-de21c269c5e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddf1cad193fa4fd184835f2ac08ae25a, user_id=884441a80e9f4d26a4fc04be59478120, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:20.721231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4627ac-88ac-44b8-81b8-de21c269c5e7 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:20.736021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1011/2009] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:20 2022] PUT /identity/v3/projects/ddf1cad193fa4fd184835f2ac08ae25a/users/884441a80e9f4d26a4fc04be59478120/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:20.746577 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c112a5-63a8-4f6a-8685-546b12634d70 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.746926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c112a5-63a8-4f6a-8685-546b12634d70 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.747668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c112a5-63a8-4f6a-8685-546b12634d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.780072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-90c112a5-63a8-4f6a-8685-546b12634d70 None None] MFA Rules not processed for user `884441a80e9f4d26a4fc04be59478120`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:20.781525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-90c112a5-63a8-4f6a-8685-546b12634d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.879600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 999/2010] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:20.886843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4627ac-88ac-44b8-81b8-de21c269c5e7 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:20.889098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4a4627ac-88ac-44b8-81b8-de21c269c5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.919342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:20.963117 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c112a5-63a8-4f6a-8685-546b12634d70 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:20.964811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-90c112a5-63a8-4f6a-8685-546b12634d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:20.977649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4627ac-88ac-44b8-81b8-de21c269c5e7 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:20.980424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:20.982687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed4c8b-eb52-4f86-8d56-0a111ad74757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:20.984015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed4c8b-eb52-4f86-8d56-0a111ad74757 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:20.984530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed4c8b-eb52-4f86-8d56-0a111ad74757 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:20.985193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed4c8b-eb52-4f86-8d56-0a111ad74757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:20.998772 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed4c8b-eb52-4f86-8d56-0a111ad74757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:21.000292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed4c8b-eb52-4f86-8d56-0a111ad74757 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:21.021916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1012/2011] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:21.045086 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed4c8b-eb52-4f86-8d56-0a111ad74757 service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:21.086618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c112a5-63a8-4f6a-8685-546b12634d70 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:21.091300 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:21.092079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.093010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.093346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] PATH_INFO: `/v3/projects/a38673eb4b9348e1b768f7e568e1fdcf` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.095208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a38673eb4b9348e1b768f7e568e1fdcf)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:21.097677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:21.155973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed4c8b-eb52-4f86-8d56-0a111ad74757 service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:21.158883 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a38673eb4b9348e1b768f7e568e1fdcf', 'request_id': 'req-164d1131-a831-41ef-99d3-f1c766e1af4e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.166971 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a38673eb4b9348e1b768f7e568e1fdcf', 'request_id': 'req-164d1131-a831-41ef-99d3-f1c766e1af4e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.174945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a38673eb4b9348e1b768f7e568e1fdcf', 'request_id': 'req-164d1131-a831-41ef-99d3-f1c766e1af4e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.191832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] The token cache is being invalidate because project a38673eb4b9348e1b768f7e568e1fdcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:21.192384 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.193240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.195102 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1000/2012] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:20 2022] DELETE /identity/v3/projects/a38673eb4b9348e1b768f7e568e1fdcf => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:21.196064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:21.198535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:21.224960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:21.225933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.226217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.226488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] PATH_INFO: `/v3/users/884441a80e9f4d26a4fc04be59478120` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.227652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=884441a80e9f4d26a4fc04be59478120)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:21.230257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:21.276875 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d1131-a831-41ef-99d3-f1c766e1af4e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:21.282704 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905cd67-b793-4fda-9b2a-ede90a30fe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:21.283349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905cd67-b793-4fda-9b2a-ede90a30fe16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.283756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905cd67-b793-4fda-9b2a-ede90a30fe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.284032 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905cd67-b793-4fda-9b2a-ede90a30fe16 admin admin] PATH_INFO: `/v3/projects/78973de997cf464e90a83cbee6f173a9` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.285537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905cd67-b793-4fda-9b2a-ede90a30fe16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78973de997cf464e90a83cbee6f173a9)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:21.287774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905cd67-b793-4fda-9b2a-ede90a30fe16 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:21.289457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '884441a80e9f4d26a4fc04be59478120'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.300209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '884441a80e9f4d26a4fc04be59478120'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.303184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '884441a80e9f4d26a4fc04be59478120'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.305248 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '884441a80e9f4d26a4fc04be59478120'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.313347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '884441a80e9f4d26a4fc04be59478120'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.318523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:21.335026 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1013/2013] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:21 2022] DELETE /identity/v3/users/884441a80e9f4d26a4fc04be59478120 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:21.336185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:21.337621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:21.359789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:21.368938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0905cd67-b793-4fda-9b2a-ede90a30fe16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78973de997cf464e90a83cbee6f173a9', 'request_id': 'req-0905cd67-b793-4fda-9b2a-ede90a30fe16'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.375276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0905cd67-b793-4fda-9b2a-ede90a30fe16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78973de997cf464e90a83cbee6f173a9', 'request_id': 'req-0905cd67-b793-4fda-9b2a-ede90a30fe16'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.381211 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0905cd67-b793-4fda-9b2a-ede90a30fe16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78973de997cf464e90a83cbee6f173a9', 'request_id': 'req-0905cd67-b793-4fda-9b2a-ede90a30fe16'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.395367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0905cd67-b793-4fda-9b2a-ede90a30fe16 admin admin] The token cache is being invalidate because project 78973de997cf464e90a83cbee6f173a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:21.395740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0905cd67-b793-4fda-9b2a-ede90a30fe16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.396489 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-0905cd67-b793-4fda-9b2a-ede90a30fe16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.398374 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1001/2014] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:21 2022] DELETE /identity/v3/projects/78973de997cf464e90a83cbee6f173a9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:21.399624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e4025-f189-44d7-8e30-f3be04dbfa91 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.400105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e4025-f189-44d7-8e30-f3be04dbfa91 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.400566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e4025-f189-44d7-8e30-f3be04dbfa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.407273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc8255-0574-4827-91b0-f10e1008e1b4 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:21.420535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8f6e4025-f189-44d7-8e30-f3be04dbfa91 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:21.421808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8f6e4025-f189-44d7-8e30-f3be04dbfa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:21.464458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5c781-6724-4ead-b9bd-e64348189b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:21.465104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5c781-6724-4ead-b9bd-e64348189b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.465385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5c781-6724-4ead-b9bd-e64348189b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.465665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5c781-6724-4ead-b9bd-e64348189b1a admin admin] PATH_INFO: `/v3/users/0a5d02feb8724947913fdbb1742d005b` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.466874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5c781-6724-4ead-b9bd-e64348189b1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a5d02feb8724947913fdbb1742d005b)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:21.469494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5c781-6724-4ead-b9bd-e64348189b1a admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:21.487809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1002/2015] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:21.487809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bdad7-05d4-4469-b56a-26f18eb225d0 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.487809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bdad7-05d4-4469-b56a-26f18eb225d0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.487809 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bdad7-05d4-4469-b56a-26f18eb225d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.507110 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e86bdad7-05d4-4469-b56a-26f18eb225d0 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:21.508325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e86bdad7-05d4-4469-b56a-26f18eb225d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:21.518837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1003/2016] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:21.520162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bdad7-05d4-4469-b56a-26f18eb225d0 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:21.520334 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-9ba5c781-6724-4ead-b9bd-e64348189b1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a5d02feb8724947913fdbb1742d005b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.521661 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e86bdad7-05d4-4469-b56a-26f18eb225d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:21.528140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-9ba5c781-6724-4ead-b9bd-e64348189b1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a5d02feb8724947913fdbb1742d005b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.529293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-9ba5c781-6724-4ead-b9bd-e64348189b1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a5d02feb8724947913fdbb1742d005b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.530595 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-9ba5c781-6724-4ead-b9bd-e64348189b1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a5d02feb8724947913fdbb1742d005b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.539460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-9ba5c781-6724-4ead-b9bd-e64348189b1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a5d02feb8724947913fdbb1742d005b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:21.564592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1014/2017] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:21 2022] DELETE /identity/v3/users/0a5d02feb8724947913fdbb1742d005b => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:21.569209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765572a5-cef1-4994-af5e-09bf252dec69 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.569571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765572a5-cef1-4994-af5e-09bf252dec69 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.569982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765572a5-cef1-4994-af5e-09bf252dec69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.596452 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-765572a5-cef1-4994-af5e-09bf252dec69 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:21.597915 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-765572a5-cef1-4994-af5e-09bf252dec69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:21.611719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:21.619831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bdad7-05d4-4469-b56a-26f18eb225d0 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:21.698207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1015/2018] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:21.703311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765572a5-cef1-4994-af5e-09bf252dec69 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:21.704635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3b50e-e42e-4d57-b883-01438694b82e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:21.705628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3b50e-e42e-4d57-b883-01438694b82e service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.706030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3b50e-e42e-4d57-b883-01438694b82e service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.706375 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3b50e-e42e-4d57-b883-01438694b82e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.708810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2be3b50e-e42e-4d57-b883-01438694b82e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:21.723825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765572a5-cef1-4994-af5e-09bf252dec69 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:21.733285 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3b50e-e42e-4d57-b883-01438694b82e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:21.734757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3b50e-e42e-4d57-b883-01438694b82e service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:21.736545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72b3e8-2241-4701-8cd0-ef1a3372e61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:21.737271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72b3e8-2241-4701-8cd0-ef1a3372e61e admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.737793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72b3e8-2241-4701-8cd0-ef1a3372e61e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.738292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d72b3e8-2241-4701-8cd0-ef1a3372e61e admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.739489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72b3e8-2241-4701-8cd0-ef1a3372e61e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:21.739870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72b3e8-2241-4701-8cd0-ef1a3372e61e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:21.742151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d72b3e8-2241-4701-8cd0-ef1a3372e61e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:21.746053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1004/2019] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:21.761060 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1016/2020] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:21.766078 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3b50e-e42e-4d57-b883-01438694b82e service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:21.776394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3b50e-e42e-4d57-b883-01438694b82e service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:21.781413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89f292-45a5-4bdb-938e-3a191973d155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:21.782135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f292-45a5-4bdb-938e-3a191973d155 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.782482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f292-45a5-4bdb-938e-3a191973d155 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.782821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f292-45a5-4bdb-938e-3a191973d155 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.783531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89f292-45a5-4bdb-938e-3a191973d155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:21.785476 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89f292-45a5-4bdb-938e-3a191973d155 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:21.809540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:21.822505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1005/2021] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:21 2022] POST /identity/v3/projects => generated 361 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:21.824267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72b3e8-2241-4701-8cd0-ef1a3372e61e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:21.841972 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d72b3e8-2241-4701-8cd0-ef1a3372e61e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:21.944724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251af642-9653-49ee-ba47-fca859ba7515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:21.945438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251af642-9653-49ee-ba47-fca859ba7515 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.945809 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251af642-9653-49ee-ba47-fca859ba7515 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.946154 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251af642-9653-49ee-ba47-fca859ba7515 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.946846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251af642-9653-49ee-ba47-fca859ba7515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:21.949127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251af642-9653-49ee-ba47-fca859ba7515 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:21.975845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1017/2022] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:21 2022] POST /identity/v3/users => generated 281 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:21.977079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89f292-45a5-4bdb-938e-3a191973d155 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:21.988281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89f292-45a5-4bdb-938e-3a191973d155 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:21.993164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67af227-3a2d-43d8-9bed-2498d8c7a05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:21.993807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67af227-3a2d-43d8-9bed-2498d8c7a05f admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.994086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67af227-3a2d-43d8-9bed-2498d8c7a05f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.994355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67af227-3a2d-43d8-9bed-2498d8c7a05f admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.995025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67af227-3a2d-43d8-9bed-2498d8c7a05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:21.996606 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67af227-3a2d-43d8-9bed-2498d8c7a05f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.008108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1006/2023] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:22.012086 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251af642-9653-49ee-ba47-fca859ba7515 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:22.022552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251af642-9653-49ee-ba47-fca859ba7515 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:22.027321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eca8d0-8f73-4505-9305-aafe9ac1e7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:22.027929 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eca8d0-8f73-4505-9305-aafe9ac1e7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.028212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eca8d0-8f73-4505-9305-aafe9ac1e7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.028485 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eca8d0-8f73-4505-9305-aafe9ac1e7f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.029261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eca8d0-8f73-4505-9305-aafe9ac1e7f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.030954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eca8d0-8f73-4505-9305-aafe9ac1e7f1 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.045991 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1018/2024] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:22.049916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67af227-3a2d-43d8-9bed-2498d8c7a05f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:22.060858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67af227-3a2d-43d8-9bed-2498d8c7a05f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:22.067619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d35a14-cecf-4804-8167-aaf2f6e21ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:22.068293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35a14-cecf-4804-8167-aaf2f6e21ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.068570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35a14-cecf-4804-8167-aaf2f6e21ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.068831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d35a14-cecf-4804-8167-aaf2f6e21ad2 admin admin] PATH_INFO: `/v3/projects/17eaebe5ce114af2b68c32195905f3a0/users/7aaed84481404064a3d1ad23958992dc/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.073910 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eca8d0-8f73-4505-9305-aafe9ac1e7f1 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:22.075293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d8eca8d0-8f73-4505-9305-aafe9ac1e7f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.088495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d35a14-cecf-4804-8167-aaf2f6e21ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17eaebe5ce114af2b68c32195905f3a0, user_id=7aaed84481404064a3d1ad23958992dc, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.090946 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eca8d0-8f73-4505-9305-aafe9ac1e7f1 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:22.091886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d35a14-cecf-4804-8167-aaf2f6e21ad2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.096420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcf83d-52bd-4102-805c-695ede9a8903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:22.097083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcf83d-52bd-4102-805c-695ede9a8903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.097606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcf83d-52bd-4102-805c-695ede9a8903 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.097942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcf83d-52bd-4102-805c-695ede9a8903 admin admin] PATH_INFO: `/v3/projects/e37118c23ace494ebb71802e04e4b4e1` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.099296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcf83d-52bd-4102-805c-695ede9a8903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e37118c23ace494ebb71802e04e4b4e1)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.101439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcf83d-52bd-4102-805c-695ede9a8903 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.116411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1007/2025] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:22 2022] PUT /identity/v3/projects/17eaebe5ce114af2b68c32195905f3a0/users/7aaed84481404064a3d1ad23958992dc/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:22.127994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc014d-8a51-4931-87aa-a46e522f105c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.128645 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc014d-8a51-4931-87aa-a46e522f105c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.128897 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc014d-8a51-4931-87aa-a46e522f105c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.160926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-45bc014d-8a51-4931-87aa-a46e522f105c None None] MFA Rules not processed for user `7aaed84481404064a3d1ad23958992dc`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:22.162081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-45bc014d-8a51-4931-87aa-a46e522f105c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.168444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5ffcf83d-52bd-4102-805c-695ede9a8903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e37118c23ace494ebb71802e04e4b4e1', 'request_id': 'req-5ffcf83d-52bd-4102-805c-695ede9a8903'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.176448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5ffcf83d-52bd-4102-805c-695ede9a8903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e37118c23ace494ebb71802e04e4b4e1', 'request_id': 'req-5ffcf83d-52bd-4102-805c-695ede9a8903'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.185239 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5ffcf83d-52bd-4102-805c-695ede9a8903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e37118c23ace494ebb71802e04e4b4e1', 'request_id': 'req-5ffcf83d-52bd-4102-805c-695ede9a8903'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.203366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5ffcf83d-52bd-4102-805c-695ede9a8903 admin admin] The token cache is being invalidate because project e37118c23ace494ebb71802e04e4b4e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:22.203586 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5ffcf83d-52bd-4102-805c-695ede9a8903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.204283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5ffcf83d-52bd-4102-805c-695ede9a8903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.206078 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1019/2026] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:22 2022] DELETE /identity/v3/projects/e37118c23ace494ebb71802e04e4b4e1 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:22.210826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3499-2642-4e46-8d3d-3772eb4df614 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.213273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3499-2642-4e46-8d3d-3772eb4df614 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.216984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3499-2642-4e46-8d3d-3772eb4df614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.225443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:22.236426 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1008/2027] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:22.237297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc014d-8a51-4931-87aa-a46e522f105c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:22.239347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-45bc014d-8a51-4931-87aa-a46e522f105c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.250580 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-420f3499-2642-4e46-8d3d-3772eb4df614 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:22.252274 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-420f3499-2642-4e46-8d3d-3772eb4df614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.328397 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1020/2028] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:22.338753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f3499-2642-4e46-8d3d-3772eb4df614 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:22.340187 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-420f3499-2642-4e46-8d3d-3772eb4df614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.354631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc014d-8a51-4931-87aa-a46e522f105c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:22.361449 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:22.362290 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.362679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.363055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] PATH_INFO: `/v3/projects/ddf1cad193fa4fd184835f2ac08ae25a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.364438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddf1cad193fa4fd184835f2ac08ae25a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.366711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.407769 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f3499-2642-4e46-8d3d-3772eb4df614 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:22.411495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b003c-5603-4957-95b0-2483aca5f9db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:22.412250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b003c-5603-4957-95b0-2483aca5f9db service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.412534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b003c-5603-4957-95b0-2483aca5f9db service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.412807 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b003c-5603-4957-95b0-2483aca5f9db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.422375 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b003c-5603-4957-95b0-2483aca5f9db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.423815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b003c-5603-4957-95b0-2483aca5f9db service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.433979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1021/2029] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:22.435429 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7ea91-094d-4773-a7a2-9c44e2a0d274 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.436047 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7ea91-094d-4773-a7a2-9c44e2a0d274 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.436047 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7ea91-094d-4773-a7a2-9c44e2a0d274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.464486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddf1cad193fa4fd184835f2ac08ae25a', 'request_id': 'req-2ca910f8-1886-4ed2-a146-79f506ebb9f1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.474715 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddf1cad193fa4fd184835f2ac08ae25a', 'request_id': 'req-2ca910f8-1886-4ed2-a146-79f506ebb9f1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.476220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-36f7ea91-094d-4773-a7a2-9c44e2a0d274 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:22.476548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:22.477441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-36f7ea91-094d-4773-a7a2-9c44e2a0d274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.487491 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddf1cad193fa4fd184835f2ac08ae25a', 'request_id': 'req-2ca910f8-1886-4ed2-a146-79f506ebb9f1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.506799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] The token cache is being invalidate because project ddf1cad193fa4fd184835f2ac08ae25a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:22.508346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.509497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.512496 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1009/2030] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:22 2022] DELETE /identity/v3/projects/ddf1cad193fa4fd184835f2ac08ae25a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:22.513651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:22.515866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.578077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1022/2031] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:22.579698 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cc224-6513-423c-9575-cf1f2e1c7709 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.579983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cc224-6513-423c-9575-cf1f2e1c7709 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.580259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cc224-6513-423c-9575-cf1f2e1c7709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.615183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca910f8-1886-4ed2-a146-79f506ebb9f1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:22.615819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-a97cc224-6513-423c-9575-cf1f2e1c7709 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:22.617036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a97cc224-6513-423c-9575-cf1f2e1c7709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.620294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:22.620990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.621276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.621548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] PATH_INFO: `/v3/users/7aaed84481404064a3d1ad23958992dc` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.622722 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aaed84481404064a3d1ad23958992dc)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.625354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.627231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1023/2032] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:22.627909 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cc224-6513-423c-9575-cf1f2e1c7709 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:22.638827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cc224-6513-423c-9575-cf1f2e1c7709 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:22.644545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd52a72-8b8b-4d0c-84c6-be955ad2bdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:22.645172 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd52a72-8b8b-4d0c-84c6-be955ad2bdaa admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.645458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd52a72-8b8b-4d0c-84c6-be955ad2bdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.645760 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd52a72-8b8b-4d0c-84c6-be955ad2bdaa admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.646557 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd52a72-8b8b-4d0c-84c6-be955ad2bdaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:22.648299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd52a72-8b8b-4d0c-84c6-be955ad2bdaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.655261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd52a72-8b8b-4d0c-84c6-be955ad2bdaa admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.672827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1024/2033] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:22.672827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fdb8f-176e-4f34-9fea-eca46f671a64 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.672827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fdb8f-176e-4f34-9fea-eca46f671a64 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.672827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75fdb8f-176e-4f34-9fea-eca46f671a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.692573 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-f75fdb8f-176e-4f34-9fea-eca46f671a64 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:22.694569 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f75fdb8f-176e-4f34-9fea-eca46f671a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.708150 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1025/2034] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:22.708150 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fdb8f-176e-4f34-9fea-eca46f671a64 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:22.710182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aaed84481404064a3d1ad23958992dc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.720362 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75fdb8f-176e-4f34-9fea-eca46f671a64 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:22.726253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a445ec5-c3c8-4b84-8e78-420cdea558fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:22.726876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a445ec5-c3c8-4b84-8e78-420cdea558fa admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.727157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a445ec5-c3c8-4b84-8e78-420cdea558fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.727429 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a445ec5-c3c8-4b84-8e78-420cdea558fa admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.728245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aaed84481404064a3d1ad23958992dc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.728367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a445ec5-c3c8-4b84-8e78-420cdea558fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.729180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aaed84481404064a3d1ad23958992dc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.729947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aaed84481404064a3d1ad23958992dc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.730343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a445ec5-c3c8-4b84-8e78-420cdea558fa admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.742457 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aaed84481404064a3d1ad23958992dc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:22.757400 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:22.785758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1026/2035] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:22 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:22.785758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a445ec5-c3c8-4b84-8e78-420cdea558fa admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:22.786176 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1010/2036] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:22 2022] DELETE /identity/v3/users/7aaed84481404064a3d1ad23958992dc => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:22.786176 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:22.787585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.788654 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8a445ec5-c3c8-4b84-8e78-420cdea558fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.869259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94154f-7c6d-44e5-a337-4b4d30e3b602 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:22.875659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd8fa1-b459-44a7-ace1-3d6ced4785ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:22.875795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a445ec5-c3c8-4b84-8e78-420cdea558fa admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:22.875990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd8fa1-b459-44a7-ace1-3d6ced4785ac admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.876268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd8fa1-b459-44a7-ace1-3d6ced4785ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.876605 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd8fa1-b459-44a7-ace1-3d6ced4785ac admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.877204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd8fa1-b459-44a7-ace1-3d6ced4785ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.879810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd8fa1-b459-44a7-ace1-3d6ced4785ac admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.884938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3de3d-5089-41b7-8fae-816b1fc4cd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:22.885564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3de3d-5089-41b7-8fae-816b1fc4cd55 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.885872 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3de3d-5089-41b7-8fae-816b1fc4cd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.886146 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3de3d-5089-41b7-8fae-816b1fc4cd55 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.886801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3de3d-5089-41b7-8fae-816b1fc4cd55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:22.887103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3de3d-5089-41b7-8fae-816b1fc4cd55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.888838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3de3d-5089-41b7-8fae-816b1fc4cd55 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.890233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:22.905810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1027/2037] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:22.905810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b9a66-4cc8-46b2-be61-5dc24db900dc None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.905810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b9a66-4cc8-46b2-be61-5dc24db900dc None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.905810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b9a66-4cc8-46b2-be61-5dc24db900dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.914997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1011/2038] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:22 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:22.916514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd8fa1-b459-44a7-ace1-3d6ced4785ac admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:22.929177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd8fa1-b459-44a7-ace1-3d6ced4785ac admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:22.935628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ba2cc-feb9-46ae-93e0-314e2b1f009d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:22.936138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-9a5b9a66-4cc8-46b2-be61-5dc24db900dc None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:22.936800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ba2cc-feb9-46ae-93e0-314e2b1f009d admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.937085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ba2cc-feb9-46ae-93e0-314e2b1f009d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.937355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ba2cc-feb9-46ae-93e0-314e2b1f009d admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.938031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ba2cc-feb9-46ae-93e0-314e2b1f009d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.938475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-9a5b9a66-4cc8-46b2-be61-5dc24db900dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.940051 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ba2cc-feb9-46ae-93e0-314e2b1f009d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.949354 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1028/2039] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:22.950390 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b9a66-4cc8-46b2-be61-5dc24db900dc None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:22.965163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b9a66-4cc8-46b2-be61-5dc24db900dc None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:22.972626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab96aa-31f2-45f0-b2f3-62bdc0a7a897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:22.973610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab96aa-31f2-45f0-b2f3-62bdc0a7a897 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.974012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab96aa-31f2-45f0-b2f3-62bdc0a7a897 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.974357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab96aa-31f2-45f0-b2f3-62bdc0a7a897 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.975074 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab96aa-31f2-45f0-b2f3-62bdc0a7a897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.976750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab96aa-31f2-45f0-b2f3-62bdc0a7a897 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.988367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1029/2040] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:22.989288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab96aa-31f2-45f0-b2f3-62bdc0a7a897 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:22.990993 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1012/2041] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:22 2022] POST /identity/v3/projects => generated 369 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:22.994346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-baab96aa-31f2-45f0-b2f3-62bdc0a7a897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.997203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ba2cc-feb9-46ae-93e0-314e2b1f009d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:23.058837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab96aa-31f2-45f0-b2f3-62bdc0a7a897 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:23.068407 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ba2cc-feb9-46ae-93e0-314e2b1f009d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:23.074246 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dabc6-b7d5-410a-9298-850e61732c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:23.074892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dabc6-b7d5-410a-9298-850e61732c0c admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.075177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dabc6-b7d5-410a-9298-850e61732c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.075505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849dabc6-b7d5-410a-9298-850e61732c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.076163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dabc6-b7d5-410a-9298-850e61732c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:23.078005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849dabc6-b7d5-410a-9298-850e61732c0c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:23.089072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1013/2042] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:23.089712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dabc6-b7d5-410a-9298-850e61732c0c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:23.100741 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849dabc6-b7d5-410a-9298-850e61732c0c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:23.106011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196af1b-2ada-4df4-b49e-23b2df19f2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:23.106672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196af1b-2ada-4df4-b49e-23b2df19f2c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.107090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196af1b-2ada-4df4-b49e-23b2df19f2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.107483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196af1b-2ada-4df4-b49e-23b2df19f2c9 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.108698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196af1b-2ada-4df4-b49e-23b2df19f2c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:23.110659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196af1b-2ada-4df4-b49e-23b2df19f2c9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:23.113147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbad41-a96a-499a-b87e-bb06449f5f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:23.114075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbad41-a96a-499a-b87e-bb06449f5f7c service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.114355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbad41-a96a-499a-b87e-bb06449f5f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.114635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fbad41-a96a-499a-b87e-bb06449f5f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.127888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbad41-a96a-499a-b87e-bb06449f5f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:23.130145 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fbad41-a96a-499a-b87e-bb06449f5f7c service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:23.137356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1014/2043] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:23 2022] POST /identity/v3/users => generated 285 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:23.137356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196af1b-2ada-4df4-b49e-23b2df19f2c9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:23.145110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1030/2044] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:23.146155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbad41-a96a-499a-b87e-bb06449f5f7c service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:23.146398 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196af1b-2ada-4df4-b49e-23b2df19f2c9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:23.151304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a154111-b755-4a8e-93f8-bd9699eb4263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:23.151910 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a154111-b755-4a8e-93f8-bd9699eb4263 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.152192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a154111-b755-4a8e-93f8-bd9699eb4263 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.152472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a154111-b755-4a8e-93f8-bd9699eb4263 admin admin] PATH_INFO: `/v3/projects/8c9621cdca494b7abf3a0e9dac3e8e9b/users/cc22b545b3fb42e2baa25cccf7891ed7/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.159520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fbad41-a96a-499a-b87e-bb06449f5f7c service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:23.165496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6a89b-07a0-4404-8cac-01bdb335800e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:23.166263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6a89b-07a0-4404-8cac-01bdb335800e admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.166629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6a89b-07a0-4404-8cac-01bdb335800e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.166999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6a89b-07a0-4404-8cac-01bdb335800e admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.167738 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6a89b-07a0-4404-8cac-01bdb335800e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:23.169442 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6a89b-07a0-4404-8cac-01bdb335800e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:23.170223 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a154111-b755-4a8e-93f8-bd9699eb4263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c9621cdca494b7abf3a0e9dac3e8e9b, user_id=cc22b545b3fb42e2baa25cccf7891ed7, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:23.173267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a154111-b755-4a8e-93f8-bd9699eb4263 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:23.186254 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1031/2045] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:23.187388 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6a89b-07a0-4404-8cac-01bdb335800e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:23.199837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1015/2046] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:23 2022] PUT /identity/v3/projects/8c9621cdca494b7abf3a0e9dac3e8e9b/users/cc22b545b3fb42e2baa25cccf7891ed7/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:23.200013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6a89b-07a0-4404-8cac-01bdb335800e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:23.201455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e21999-5828-4ea1-97e7-31e5afd1866a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.201873 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e21999-5828-4ea1-97e7-31e5afd1866a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.202230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e21999-5828-4ea1-97e7-31e5afd1866a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.217921 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:23.233852 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-64e21999-5828-4ea1-97e7-31e5afd1866a None None] MFA Rules not processed for user `cc22b545b3fb42e2baa25cccf7891ed7`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:23.235204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-64e21999-5828-4ea1-97e7-31e5afd1866a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:23.261763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd8584-42fd-4ca7-9717-50dcc59c34cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:23.262498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd8584-42fd-4ca7-9717-50dcc59c34cc admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.262851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd8584-42fd-4ca7-9717-50dcc59c34cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.263197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd8584-42fd-4ca7-9717-50dcc59c34cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.264017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd8584-42fd-4ca7-9717-50dcc59c34cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:23.265722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd8584-42fd-4ca7-9717-50dcc59c34cc admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:23.276723 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1032/2047] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:23.281346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd8584-42fd-4ca7-9717-50dcc59c34cc admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:23.291870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd8584-42fd-4ca7-9717-50dcc59c34cc admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:23.298065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c5733-84a2-4338-9ce4-ad3d44f1f9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:23.298728 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5733-84a2-4338-9ce4-ad3d44f1f9d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.299044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5733-84a2-4338-9ce4-ad3d44f1f9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.299283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5733-84a2-4338-9ce4-ad3d44f1f9d6 admin admin] PATH_INFO: `/v3/projects/c0846ec480d240c98847746726f43813/users/208a61853ba047028e682f4adfd65eca/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.315623 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1016/2048] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:23.320211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c5733-84a2-4338-9ce4-ad3d44f1f9d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0846ec480d240c98847746726f43813, user_id=208a61853ba047028e682f4adfd65eca, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:23.323670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c5733-84a2-4338-9ce4-ad3d44f1f9d6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:23.326103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e21999-5828-4ea1-97e7-31e5afd1866a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:23.327957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-64e21999-5828-4ea1-97e7-31e5afd1866a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:23.343207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1033/2049] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:23 2022] PUT /identity/v3/projects/c0846ec480d240c98847746726f43813/users/208a61853ba047028e682f4adfd65eca/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:23.352690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:23.353566 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c9b48-b4d6-47c6-916e-939034e1e698 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.353891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c9b48-b4d6-47c6-916e-939034e1e698 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.354174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7c9b48-b4d6-47c6-916e-939034e1e698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.385776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-3e7c9b48-b4d6-47c6-916e-939034e1e698 None None] MFA Rules not processed for user `208a61853ba047028e682f4adfd65eca`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:23.387063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-3e7c9b48-b4d6-47c6-916e-939034e1e698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:23.414332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e21999-5828-4ea1-97e7-31e5afd1866a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:23.482366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226acad8-ec29-4134-9964-fff61e4f8796 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:23.483044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226acad8-ec29-4134-9964-fff61e4f8796 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.486284 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1034/2050] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:23.487268 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c9b48-b4d6-47c6-916e-939034e1e698 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:23.488762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-3e7c9b48-b4d6-47c6-916e-939034e1e698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:23.495491 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226acad8-ec29-4134-9964-fff61e4f8796 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.495970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226acad8-ec29-4134-9964-fff61e4f8796 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.567544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7c9b48-b4d6-47c6-916e-939034e1e698 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:23.572379 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1772cd-46fd-43f7-9905-c45b3a607587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:23.573039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1772cd-46fd-43f7-9905-c45b3a607587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.573322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1772cd-46fd-43f7-9905-c45b3a607587 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.573595 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1772cd-46fd-43f7-9905-c45b3a607587 admin admin] PATH_INFO: `/v3/projects/17eaebe5ce114af2b68c32195905f3a0` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.574752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1772cd-46fd-43f7-9905-c45b3a607587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17eaebe5ce114af2b68c32195905f3a0)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:23.576692 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1772cd-46fd-43f7-9905-c45b3a607587 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:23.577803 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226acad8-ec29-4134-9964-fff61e4f8796 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:23.579237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226acad8-ec29-4134-9964-fff61e4f8796 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:23.588921 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1017/2051] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:23.589755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226acad8-ec29-4134-9964-fff61e4f8796 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:23.591063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-226acad8-ec29-4134-9964-fff61e4f8796 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:23.616635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:23.651421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-0e1772cd-46fd-43f7-9905-c45b3a607587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17eaebe5ce114af2b68c32195905f3a0', 'request_id': 'req-0e1772cd-46fd-43f7-9905-c45b3a607587'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:23.674850 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-0e1772cd-46fd-43f7-9905-c45b3a607587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17eaebe5ce114af2b68c32195905f3a0', 'request_id': 'req-0e1772cd-46fd-43f7-9905-c45b3a607587'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:23.687419 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-0e1772cd-46fd-43f7-9905-c45b3a607587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17eaebe5ce114af2b68c32195905f3a0', 'request_id': 'req-0e1772cd-46fd-43f7-9905-c45b3a607587'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:23.701575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-0e1772cd-46fd-43f7-9905-c45b3a607587 admin admin] The token cache is being invalidate because project 17eaebe5ce114af2b68c32195905f3a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:23.702694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-0e1772cd-46fd-43f7-9905-c45b3a607587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:23.703543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-0e1772cd-46fd-43f7-9905-c45b3a607587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:23.708691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1035/2052] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:23 2022] DELETE /identity/v3/projects/17eaebe5ce114af2b68c32195905f3a0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:23.710166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781be3ac-a19b-4864-8f70-c0a2c55f626e None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.710454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781be3ac-a19b-4864-8f70-c0a2c55f626e None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.710732 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781be3ac-a19b-4864-8f70-c0a2c55f626e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.710836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226acad8-ec29-4134-9964-fff61e4f8796 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:23.738055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:23.738523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-781be3ac-a19b-4864-8f70-c0a2c55f626e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:23.739959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-781be3ac-a19b-4864-8f70-c0a2c55f626e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:23.788137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da9852-506f-41f1-8886-01fa30d3f9e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e27f5b81c704dbebbd6a83b2713e237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:23.788874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da9852-506f-41f1-8886-01fa30d3f9e1 service glance] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.789223 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da9852-506f-41f1-8886-01fa30d3f9e1 service glance] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.789726 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da9852-506f-41f1-8886-01fa30d3f9e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.792708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-34da9852-506f-41f1-8886-01fa30d3f9e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:23.848088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1036/2053] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:23.849990 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c8e1c-e1c4-4ef0-9b87-b13e7a00fe30 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.850280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c8e1c-e1c4-4ef0-9b87-b13e7a00fe30 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.850756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170c8e1c-e1c4-4ef0-9b87-b13e7a00fe30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.880968 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-170c8e1c-e1c4-4ef0-9b87-b13e7a00fe30 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:23.882243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-170c8e1c-e1c4-4ef0-9b87-b13e7a00fe30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:23.893484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1037/2054] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:23.901657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da9852-506f-41f1-8886-01fa30d3f9e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:23.904434 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da9852-506f-41f1-8886-01fa30d3f9e1 service glance] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:23.914406 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c8e1c-e1c4-4ef0-9b87-b13e7a00fe30 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:23.929504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1018/2055] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:23.944759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170c8e1c-e1c4-4ef0-9b87-b13e7a00fe30 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:23.963256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adac35-5a6c-4f68-9f87-a98b53ae376a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:23.963256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adac35-5a6c-4f68-9f87-a98b53ae376a admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:23.963256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adac35-5a6c-4f68-9f87-a98b53ae376a admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:23.963256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adac35-5a6c-4f68-9f87-a98b53ae376a admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:23.963590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adac35-5a6c-4f68-9f87-a98b53ae376a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:23.963590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adac35-5a6c-4f68-9f87-a98b53ae376a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:23.966593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31adac35-5a6c-4f68-9f87-a98b53ae376a admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:23.981107 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1038/2056] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:23.987010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da9852-506f-41f1-8886-01fa30d3f9e1 service glance] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:23.999865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da9852-506f-41f1-8886-01fa30d3f9e1 service glance] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:24.004678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061f791-f579-4bb7-b67f-79481360a1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:24.005293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061f791-f579-4bb7-b67f-79481360a1bc admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:24.005573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061f791-f579-4bb7-b67f-79481360a1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:24.005885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061f791-f579-4bb7-b67f-79481360a1bc admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:24.006504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061f791-f579-4bb7-b67f-79481360a1bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:24.008538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061f791-f579-4bb7-b67f-79481360a1bc admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:24.054022 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1019/2057] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:23 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:24.054923 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adac35-5a6c-4f68-9f87-a98b53ae376a admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:24.072576 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31adac35-5a6c-4f68-9f87-a98b53ae376a admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:24.223569 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f9dbb-4d64-41ad-bdfd-ac38d60a8cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:24.224121 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f9dbb-4d64-41ad-bdfd-ac38d60a8cdc admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:24.224403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f9dbb-4d64-41ad-bdfd-ac38d60a8cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:24.224676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f9dbb-4d64-41ad-bdfd-ac38d60a8cdc admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:24.225463 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f9dbb-4d64-41ad-bdfd-ac38d60a8cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:24.227417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f9dbb-4d64-41ad-bdfd-ac38d60a8cdc admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:24.289835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1039/2058] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:24 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:24.294302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061f791-f579-4bb7-b67f-79481360a1bc admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:24.311427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061f791-f579-4bb7-b67f-79481360a1bc admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:24.323997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8aef6-2273-4a1a-9f67-7992b7d0174a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:24.323997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8aef6-2273-4a1a-9f67-7992b7d0174a admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:24.324284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8aef6-2273-4a1a-9f67-7992b7d0174a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:24.324499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8aef6-2273-4a1a-9f67-7992b7d0174a admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:24.325630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8aef6-2273-4a1a-9f67-7992b7d0174a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:24.328187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8aef6-2273-4a1a-9f67-7992b7d0174a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:24.350872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1020/2059] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:24.352919 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f9dbb-4d64-41ad-bdfd-ac38d60a8cdc admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:24.370439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f9dbb-4d64-41ad-bdfd-ac38d60a8cdc admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:24.383104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498502b-1846-4b41-9d56-0b6a8ff9f1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:24.383727 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498502b-1846-4b41-9d56-0b6a8ff9f1de admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:24.384003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498502b-1846-4b41-9d56-0b6a8ff9f1de admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:24.384274 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498502b-1846-4b41-9d56-0b6a8ff9f1de admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:24.384929 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498502b-1846-4b41-9d56-0b6a8ff9f1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:24.392960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498502b-1846-4b41-9d56-0b6a8ff9f1de admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:24.416664 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1040/2060] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:24.418596 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8aef6-2273-4a1a-9f67-7992b7d0174a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:24.454152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8aef6-2273-4a1a-9f67-7992b7d0174a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:24.476615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d141ebb-9abe-4792-8382-ab3787c788cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:24.477373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d141ebb-9abe-4792-8382-ab3787c788cd admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:24.477664 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d141ebb-9abe-4792-8382-ab3787c788cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:24.478115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d141ebb-9abe-4792-8382-ab3787c788cd admin admin] PATH_INFO: `/v3/projects/6e573243b2a14e24a00aa569a5dd890b/users/f4e1eb3d9aa64115ab90da7b8a700a16/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:24.535977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d141ebb-9abe-4792-8382-ab3787c788cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e573243b2a14e24a00aa569a5dd890b, user_id=f4e1eb3d9aa64115ab90da7b8a700a16, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:24.538756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d141ebb-9abe-4792-8382-ab3787c788cd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:24.568089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1021/2061] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:24 2022] PUT /identity/v3/projects/6e573243b2a14e24a00aa569a5dd890b/users/f4e1eb3d9aa64115ab90da7b8a700a16/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:24.581373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250687b8-00b2-48c2-8d0f-e6bf03df08bd None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:24.581801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250687b8-00b2-48c2-8d0f-e6bf03df08bd None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:24.582316 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250687b8-00b2-48c2-8d0f-e6bf03df08bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:24.625612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:24.635503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-250687b8-00b2-48c2-8d0f-e6bf03df08bd None None] MFA Rules not processed for user `f4e1eb3d9aa64115ab90da7b8a700a16`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:24.639041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-250687b8-00b2-48c2-8d0f-e6bf03df08bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:24.761890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1041/2062] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:24.772252 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d141ebb-9abe-4792-8382-ab3787c788cd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:24.774316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2d141ebb-9abe-4792-8382-ab3787c788cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:24.844228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d141ebb-9abe-4792-8382-ab3787c788cd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:24.848741 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cb473-6eec-4d42-bfcc-b83281623345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:24.849566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cb473-6eec-4d42-bfcc-b83281623345 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:24.850014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cb473-6eec-4d42-bfcc-b83281623345 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:24.850296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114cb473-6eec-4d42-bfcc-b83281623345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:24.859650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cb473-6eec-4d42-bfcc-b83281623345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:24.861128 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114cb473-6eec-4d42-bfcc-b83281623345 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:24.870784 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1022/2063] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:27.693956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250687b8-00b2-48c2-8d0f-e6bf03df08bd None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.695889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-250687b8-00b2-48c2-8d0f-e6bf03df08bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:27.716659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cb473-6eec-4d42-bfcc-b83281623345 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:27.740242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:27.805958 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250687b8-00b2-48c2-8d0f-e6bf03df08bd None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.807588 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114cb473-6eec-4d42-bfcc-b83281623345 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:27.818089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def964f-0d5e-4adc-ae2d-2355e2d86acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.819024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def964f-0d5e-4adc-ae2d-2355e2d86acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.819459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def964f-0d5e-4adc-ae2d-2355e2d86acb admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.819587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aec5df-12e8-4595-bd12-a1e726631c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:27.820063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def964f-0d5e-4adc-ae2d-2355e2d86acb admin admin] PATH_INFO: `/v3/users/cc22b545b3fb42e2baa25cccf7891ed7` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.820195 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aec5df-12e8-4595-bd12-a1e726631c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.820386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aec5df-12e8-4595-bd12-a1e726631c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.820684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aec5df-12e8-4595-bd12-a1e726631c68 admin admin] PATH_INFO: `/v3/users/f4e1eb3d9aa64115ab90da7b8a700a16` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.821713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def964f-0d5e-4adc-ae2d-2355e2d86acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc22b545b3fb42e2baa25cccf7891ed7)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.822046 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aec5df-12e8-4595-bd12-a1e726631c68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4e1eb3d9aa64115ab90da7b8a700a16)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:27.823971 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aec5df-12e8-4595-bd12-a1e726631c68 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.824186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def964f-0d5e-4adc-ae2d-2355e2d86acb admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:27.887261 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a2aec5df-12e8-4595-bd12-a1e726631c68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4e1eb3d9aa64115ab90da7b8a700a16'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:27.908561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a2aec5df-12e8-4595-bd12-a1e726631c68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4e1eb3d9aa64115ab90da7b8a700a16'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:27.909344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a2aec5df-12e8-4595-bd12-a1e726631c68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4e1eb3d9aa64115ab90da7b8a700a16'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:27.910405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a2aec5df-12e8-4595-bd12-a1e726631c68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4e1eb3d9aa64115ab90da7b8a700a16'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:27.918495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5def964f-0d5e-4adc-ae2d-2355e2d86acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc22b545b3fb42e2baa25cccf7891ed7'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:27.924238 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a2aec5df-12e8-4595-bd12-a1e726631c68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4e1eb3d9aa64115ab90da7b8a700a16'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:27.930395 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5def964f-0d5e-4adc-ae2d-2355e2d86acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc22b545b3fb42e2baa25cccf7891ed7'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:27.932384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5def964f-0d5e-4adc-ae2d-2355e2d86acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc22b545b3fb42e2baa25cccf7891ed7'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:27.933661 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5def964f-0d5e-4adc-ae2d-2355e2d86acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc22b545b3fb42e2baa25cccf7891ed7'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:27.943781 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5def964f-0d5e-4adc-ae2d-2355e2d86acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc22b545b3fb42e2baa25cccf7891ed7'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:27.968864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1042/2064] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:27 2022] DELETE /identity/v3/users/cc22b545b3fb42e2baa25cccf7891ed7 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:27.973220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8c1d9-dd50-45c7-80f1-e0ab948925e2 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.973512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8c1d9-dd50-45c7-80f1-e0ab948925e2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.973952 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8c1d9-dd50-45c7-80f1-e0ab948925e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.987177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1023/2065] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:27 2022] DELETE /identity/v3/users/f4e1eb3d9aa64115ab90da7b8a700a16 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:27.993373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdc939-3ba2-44d5-a2a8-f2a1740424dc None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:27.994331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdc939-3ba2-44d5-a2a8-f2a1740424dc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:27.994779 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdc939-3ba2-44d5-a2a8-f2a1740424dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:27.997913 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-58c8c1d9-dd50-45c7-80f1-e0ab948925e2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:27.999730 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-58c8c1d9-dd50-45c7-80f1-e0ab948925e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.019453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-defdc939-3ba2-44d5-a2a8-f2a1740424dc None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:28.021770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-defdc939-3ba2-44d5-a2a8-f2a1740424dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.025800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:28.073758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1043/2066] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:28.084099 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8c1d9-dd50-45c7-80f1-e0ab948925e2 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.086932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-58c8c1d9-dd50-45c7-80f1-e0ab948925e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.096135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1024/2067] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:28.107685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdc939-3ba2-44d5-a2a8-f2a1740424dc None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.111215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-defdc939-3ba2-44d5-a2a8-f2a1740424dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.194129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8c1d9-dd50-45c7-80f1-e0ab948925e2 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.194980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:28.199581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a6719-d4a8-496a-8bde-3666b1cac3d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.200215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a6719-d4a8-496a-8bde-3666b1cac3d8 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.200628 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a6719-d4a8-496a-8bde-3666b1cac3d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.201030 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a6719-d4a8-496a-8bde-3666b1cac3d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.212516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a6719-d4a8-496a-8bde-3666b1cac3d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.214343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a6719-d4a8-496a-8bde-3666b1cac3d8 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.216108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdc939-3ba2-44d5-a2a8-f2a1740424dc None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.220638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562780b8-ba61-4e9c-a10b-ba5482691b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.221572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562780b8-ba61-4e9c-a10b-ba5482691b86 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.222290 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562780b8-ba61-4e9c-a10b-ba5482691b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.222814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562780b8-ba61-4e9c-a10b-ba5482691b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.229109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1044/2068] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:28.235381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562780b8-ba61-4e9c-a10b-ba5482691b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.237212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562780b8-ba61-4e9c-a10b-ba5482691b86 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.258961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1025/2069] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:28.618540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a6719-d4a8-496a-8bde-3666b1cac3d8 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.620393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e53a6719-d4a8-496a-8bde-3666b1cac3d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.647875 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a6719-d4a8-496a-8bde-3666b1cac3d8 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.678382 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.684308 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.686186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.688809 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] PATH_INFO: `/v3/users/d65c95fd491f43aab82137b25aa0610d` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.691094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d65c95fd491f43aab82137b25aa0610d)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.696546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:28.766999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd65c95fd491f43aab82137b25aa0610d'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:28.782512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd65c95fd491f43aab82137b25aa0610d'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:28.783366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd65c95fd491f43aab82137b25aa0610d'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:28.784487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd65c95fd491f43aab82137b25aa0610d'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:28.792849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd65c95fd491f43aab82137b25aa0610d'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:28.825427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1045/2070] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:28 2022] DELETE /identity/v3/users/d65c95fd491f43aab82137b25aa0610d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:28.827774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a7759-c49d-4c46-98a7-0ae2456f0778 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.828269 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a7759-c49d-4c46-98a7-0ae2456f0778 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.828853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a7759-c49d-4c46-98a7-0ae2456f0778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.860337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-952a7759-c49d-4c46-98a7-0ae2456f0778 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:28.863300 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-952a7759-c49d-4c46-98a7-0ae2456f0778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.902539 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.913226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.966339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1026/2071] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:28.968057 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a7759-c49d-4c46-98a7-0ae2456f0778 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:28.969603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-952a7759-c49d-4c46-98a7-0ae2456f0778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.983327 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a7759-c49d-4c46-98a7-0ae2456f0778 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:28.989097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:28.989794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.990066 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.990333 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] PATH_INFO: `/v3/projects/6e573243b2a14e24a00aa569a5dd890b` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.992517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e573243b2a14e24a00aa569a5dd890b)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:28.994851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.003092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:29.037786 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f29df-afed-4bb2-b2cb-a3369c1deb2e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.048293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.049178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.050561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.050561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8 admin admin] PATH_INFO: `/v3/projects/8c9621cdca494b7abf3a0e9dac3e8e9b` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.054396 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c9621cdca494b7abf3a0e9dac3e8e9b)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.056987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.073943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e573243b2a14e24a00aa569a5dd890b', 'request_id': 'req-803d9e68-1081-4847-acbe-73e9bbb72877'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:29.073943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e573243b2a14e24a00aa569a5dd890b', 'request_id': 'req-803d9e68-1081-4847-acbe-73e9bbb72877'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:29.075393 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e573243b2a14e24a00aa569a5dd890b', 'request_id': 'req-803d9e68-1081-4847-acbe-73e9bbb72877'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:29.090512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] The token cache is being invalidate because project 6e573243b2a14e24a00aa569a5dd890b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:29.091261 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:29.094332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:29.100231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1027/2072] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:28 2022] DELETE /identity/v3/projects/6e573243b2a14e24a00aa569a5dd890b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:29.100231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.100231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.135712 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c9621cdca494b7abf3a0e9dac3e8e9b', 'request_id': 'req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:29.144171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c9621cdca494b7abf3a0e9dac3e8e9b', 'request_id': 'req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:29.152408 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c9621cdca494b7abf3a0e9dac3e8e9b', 'request_id': 'req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:29.164090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8 admin admin] The token cache is being invalidate because project 8c9621cdca494b7abf3a0e9dac3e8e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:29.164486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:29.165234 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-30a5c9b0-3b39-4a9a-9e3a-f16f139055a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:29.167559 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1046/2073] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:28 2022] DELETE /identity/v3/projects/8c9621cdca494b7abf3a0e9dac3e8e9b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:29.171217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbc281-71c0-47b5-9a84-01de565f726c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.171217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbc281-71c0-47b5-9a84-01de565f726c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.171217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbc281-71c0-47b5-9a84-01de565f726c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.175758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d9e68-1081-4847-acbe-73e9bbb72877 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.193801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-36fbc281-71c0-47b5-9a84-01de565f726c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:29.194810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-36fbc281-71c0-47b5-9a84-01de565f726c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.200828 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:29.244547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05295b09-6316-4803-8d5f-f1d5bd849d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.245522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05295b09-6316-4803-8d5f-f1d5bd849d1a service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.245853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05295b09-6316-4803-8d5f-f1d5bd849d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.246131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05295b09-6316-4803-8d5f-f1d5bd849d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.247907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-05295b09-6316-4803-8d5f-f1d5bd849d1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.264224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1047/2074] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:29.265536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246f951-87cb-49cd-8ea0-d60347b07047 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.266018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246f951-87cb-49cd-8ea0-d60347b07047 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.266348 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246f951-87cb-49cd-8ea0-d60347b07047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.289518 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-f246f951-87cb-49cd-8ea0-d60347b07047 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:29.291072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f246f951-87cb-49cd-8ea0-d60347b07047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.300350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1048/2075] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:29.303226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2145c3-7399-4e05-819b-b5ede8701f75 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.306260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2145c3-7399-4e05-819b-b5ede8701f75 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.306778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2145c3-7399-4e05-819b-b5ede8701f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.330102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-7f2145c3-7399-4e05-819b-b5ede8701f75 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:29.331227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7f2145c3-7399-4e05-819b-b5ede8701f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.336672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05295b09-6316-4803-8d5f-f1d5bd849d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.338274 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05295b09-6316-4803-8d5f-f1d5bd849d1a service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.344706 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1049/2076] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:29.346284 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e330f6-34c2-4f70-b8fa-404eeac5e157 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.346571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e330f6-34c2-4f70-b8fa-404eeac5e157 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.346845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e330f6-34c2-4f70-b8fa-404eeac5e157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.349726 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1028/2077] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:29.350568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05295b09-6316-4803-8d5f-f1d5bd849d1a service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.361154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05295b09-6316-4803-8d5f-f1d5bd849d1a service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.366493 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114034c9-65b4-4029-b4d3-af92a4b23b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.367184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114034c9-65b4-4029-b4d3-af92a4b23b6a admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.367538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114034c9-65b4-4029-b4d3-af92a4b23b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.367903 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114034c9-65b4-4029-b4d3-af92a4b23b6a admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.368645 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114034c9-65b4-4029-b4d3-af92a4b23b6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:29.369003 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114034c9-65b4-4029-b4d3-af92a4b23b6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.370693 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114034c9-65b4-4029-b4d3-af92a4b23b6a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.378686 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-29e330f6-34c2-4f70-b8fa-404eeac5e157 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:29.379949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-29e330f6-34c2-4f70-b8fa-404eeac5e157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.387013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1029/2078] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:29.389922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114034c9-65b4-4029-b4d3-af92a4b23b6a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.396199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1050/2079] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:29.397328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e330f6-34c2-4f70-b8fa-404eeac5e157 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.401301 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114034c9-65b4-4029-b4d3-af92a4b23b6a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.407592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e330f6-34c2-4f70-b8fa-404eeac5e157 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.408481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df3aee-4726-47bb-a4d8-491cd01ae33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.409783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df3aee-4726-47bb-a4d8-491cd01ae33c admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.410070 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df3aee-4726-47bb-a4d8-491cd01ae33c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.410339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df3aee-4726-47bb-a4d8-491cd01ae33c admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.410983 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df3aee-4726-47bb-a4d8-491cd01ae33c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.413183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df3aee-4726-47bb-a4d8-491cd01ae33c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.413982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684d29b-f1b1-4e8d-927f-fc8812710e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.414976 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684d29b-f1b1-4e8d-927f-fc8812710e2c admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.415438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684d29b-f1b1-4e8d-927f-fc8812710e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.415965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0684d29b-f1b1-4e8d-927f-fc8812710e2c admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.416660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684d29b-f1b1-4e8d-927f-fc8812710e2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:29.416967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684d29b-f1b1-4e8d-927f-fc8812710e2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.418797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0684d29b-f1b1-4e8d-927f-fc8812710e2c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.433720 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1051/2080] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:29.438257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684d29b-f1b1-4e8d-927f-fc8812710e2c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.454963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1030/2081] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:29.456529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0684d29b-f1b1-4e8d-927f-fc8812710e2c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.460103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df3aee-4726-47bb-a4d8-491cd01ae33c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.503830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:29.544981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b9428-33ef-4556-b266-0c46f6231789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.546391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b9428-33ef-4556-b266-0c46f6231789 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.546732 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b9428-33ef-4556-b266-0c46f6231789 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.546948 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b9428-33ef-4556-b266-0c46f6231789 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.547886 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b9428-33ef-4556-b266-0c46f6231789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.549850 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b9428-33ef-4556-b266-0c46f6231789 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.562599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df3aee-4726-47bb-a4d8-491cd01ae33c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.568133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67140bf3-b1ca-4ba3-8ab9-1f9a7d795be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.568761 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67140bf3-b1ca-4ba3-8ab9-1f9a7d795be3 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.569168 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67140bf3-b1ca-4ba3-8ab9-1f9a7d795be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.569474 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67140bf3-b1ca-4ba3-8ab9-1f9a7d795be3 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.570505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67140bf3-b1ca-4ba3-8ab9-1f9a7d795be3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.572275 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67140bf3-b1ca-4ba3-8ab9-1f9a7d795be3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.584680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1052/2082] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:29.590031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b9428-33ef-4556-b266-0c46f6231789 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.618133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1031/2083] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:29.619050 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67140bf3-b1ca-4ba3-8ab9-1f9a7d795be3 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.635041 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67140bf3-b1ca-4ba3-8ab9-1f9a7d795be3 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.672026 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:29.719217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baee35-5057-42e6-a1bd-36e5815b1476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.719755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baee35-5057-42e6-a1bd-36e5815b1476 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.720036 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baee35-5057-42e6-a1bd-36e5815b1476 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.720314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baee35-5057-42e6-a1bd-36e5815b1476 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.723016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baee35-5057-42e6-a1bd-36e5815b1476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.723016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baee35-5057-42e6-a1bd-36e5815b1476 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.730748 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b9428-33ef-4556-b266-0c46f6231789 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.736152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bb28b-6398-4ed7-a912-865a94c6fc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.736764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bb28b-6398-4ed7-a912-865a94c6fc17 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.737353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bb28b-6398-4ed7-a912-865a94c6fc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.737353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bb28b-6398-4ed7-a912-865a94c6fc17 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.738140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bb28b-6398-4ed7-a912-865a94c6fc17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.738248 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1032/2084] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:29.740025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bb28b-6398-4ed7-a912-865a94c6fc17 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.742051 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baee35-5057-42e6-a1bd-36e5815b1476 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.752627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baee35-5057-42e6-a1bd-36e5815b1476 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.759110 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d908f41-8a67-4ed8-a86d-073704424377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.759901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d908f41-8a67-4ed8-a86d-073704424377 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.761249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d908f41-8a67-4ed8-a86d-073704424377 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.761849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d908f41-8a67-4ed8-a86d-073704424377 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.762548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d908f41-8a67-4ed8-a86d-073704424377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.764691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d908f41-8a67-4ed8-a86d-073704424377 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.770642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1053/2085] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:29.774775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bb28b-6398-4ed7-a912-865a94c6fc17 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.776529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1033/2086] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:29.780536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d908f41-8a67-4ed8-a86d-073704424377 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.787595 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bb28b-6398-4ed7-a912-865a94c6fc17 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.791439 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d908f41-8a67-4ed8-a86d-073704424377 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.795038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a1afa-47d5-4c5c-9119-025a7f62c6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.795907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a1afa-47d5-4c5c-9119-025a7f62c6ba admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.796182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a1afa-47d5-4c5c-9119-025a7f62c6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.796449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a1afa-47d5-4c5c-9119-025a7f62c6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.797788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a1afa-47d5-4c5c-9119-025a7f62c6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.798267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd361053-7590-4b04-bd6b-f3e5877aad58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.798938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd361053-7590-4b04-bd6b-f3e5877aad58 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.799750 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd361053-7590-4b04-bd6b-f3e5877aad58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.799859 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a1afa-47d5-4c5c-9119-025a7f62c6ba admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.799953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd361053-7590-4b04-bd6b-f3e5877aad58 admin admin] PATH_INFO: `/v3/projects/f7507a7da3bb4420abcfcf08780dc0a4/users/bed19e9a74be46b5bc836c3cd605133f/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.812402 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1054/2087] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:29.816571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a1afa-47d5-4c5c-9119-025a7f62c6ba admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.818364 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd361053-7590-4b04-bd6b-f3e5877aad58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7507a7da3bb4420abcfcf08780dc0a4, user_id=bed19e9a74be46b5bc836c3cd605133f, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.821126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd361053-7590-4b04-bd6b-f3e5877aad58 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.829828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a1afa-47d5-4c5c-9119-025a7f62c6ba admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.836456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1034/2088] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:29 2022] PUT /identity/v3/projects/f7507a7da3bb4420abcfcf08780dc0a4/users/bed19e9a74be46b5bc836c3cd605133f/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:29.842926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887e932-b590-433b-bb9a-e685b4dee201 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.843257 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887e932-b590-433b-bb9a-e685b4dee201 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.843788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887e932-b590-433b-bb9a-e685b4dee201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.870406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0887e932-b590-433b-bb9a-e685b4dee201 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:29.871836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0887e932-b590-433b-bb9a-e685b4dee201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.938588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3947db5-7739-4e06-9fa0-6b39bc36a331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.939283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3947db5-7739-4e06-9fa0-6b39bc36a331 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.939632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3947db5-7739-4e06-9fa0-6b39bc36a331 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.939970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3947db5-7739-4e06-9fa0-6b39bc36a331 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.940673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3947db5-7739-4e06-9fa0-6b39bc36a331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.942426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3947db5-7739-4e06-9fa0-6b39bc36a331 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:29.946363 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1035/2089] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:29.950688 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887e932-b590-433b-bb9a-e685b4dee201 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.954074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0887e932-b590-433b-bb9a-e685b4dee201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.960391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1055/2090] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:29.961466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3947db5-7739-4e06-9fa0-6b39bc36a331 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:29.965198 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a3947db5-7739-4e06-9fa0-6b39bc36a331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:29.967608 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887e932-b590-433b-bb9a-e685b4dee201 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:29.972202 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:29.972766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:29.973038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:29.973306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] PATH_INFO: `/v3/projects/73e9543ac314447194b3a4d851a65bf0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:29.974420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] RBAC: Authorizing `identity:delete_project(project_id=73e9543ac314447194b3a4d851a65bf0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:29.976307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:30.007766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:30.024889 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73e9543ac314447194b3a4d851a65bf0', 'request_id': 'req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:30.028757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3947db5-7739-4e06-9fa0-6b39bc36a331 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:30.032213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73e9543ac314447194b3a4d851a65bf0', 'request_id': 'req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:30.038231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73e9543ac314447194b3a4d851a65bf0', 'request_id': 'req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:30.051494 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] The token cache is being invalidate because project 73e9543ac314447194b3a4d851a65bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:30.051789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:30.052447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:30.054061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1036/2091] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:29 2022] DELETE /identity/v3/projects/73e9543ac314447194b3a4d851a65bf0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:30.054938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:30.056252 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:30.069757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:30.081436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de21ca9-b5ec-42ff-8ac3-3ebd20f9dbdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:30.081791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de21ca9-b5ec-42ff-8ac3-3ebd20f9dbdc service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:30.082079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de21ca9-b5ec-42ff-8ac3-3ebd20f9dbdc service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:30.082437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de21ca9-b5ec-42ff-8ac3-3ebd20f9dbdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:30.084316 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7de21ca9-b5ec-42ff-8ac3-3ebd20f9dbdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:30.126025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba085b-51a1-4b25-92d4-2c3f0561bbab admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:30.130811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247cb74-846d-4cdc-91c4-7a3fdbf9744e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:30.131480 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247cb74-846d-4cdc-91c4-7a3fdbf9744e admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:30.131762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247cb74-846d-4cdc-91c4-7a3fdbf9744e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:30.132040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247cb74-846d-4cdc-91c4-7a3fdbf9744e admin admin] PATH_INFO: `/v3/projects/06929ea6da40417394a9274b2c86c96a/users/e60401b675bb4dff9ebc12b6076fdd3f/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:30.148166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247cb74-846d-4cdc-91c4-7a3fdbf9744e admin admin] RBAC: Authorizing `identity:create_grant(project_id=06929ea6da40417394a9274b2c86c96a, user_id=e60401b675bb4dff9ebc12b6076fdd3f, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:30.150902 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247cb74-846d-4cdc-91c4-7a3fdbf9744e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:30.161055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de21ca9-b5ec-42ff-8ac3-3ebd20f9dbdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:30.162568 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de21ca9-b5ec-42ff-8ac3-3ebd20f9dbdc service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:30.165336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1037/2092] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:30 2022] PUT /identity/v3/projects/06929ea6da40417394a9274b2c86c96a/users/e60401b675bb4dff9ebc12b6076fdd3f/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:30.166653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247cb74-846d-4cdc-91c4-7a3fdbf9744e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:30.174589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9247cb74-846d-4cdc-91c4-7a3fdbf9744e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:30.179505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1056/2093] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:30.181155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22262807-e312-4d0f-b882-fb290853325c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:30.181448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22262807-e312-4d0f-b882-fb290853325c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:30.181724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22262807-e312-4d0f-b882-fb290853325c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:30.224574 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-22262807-e312-4d0f-b882-fb290853325c None None] MFA Rules not processed for user `e60401b675bb4dff9ebc12b6076fdd3f`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:30.225710 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-22262807-e312-4d0f-b882-fb290853325c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:30.256370 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247cb74-846d-4cdc-91c4-7a3fdbf9744e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:30.261544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:30.262438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:30.262972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:30.263115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] PATH_INFO: `/v3/users/8e3bb221e0db476280f7e49820880544` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:30.264472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e3bb221e0db476280f7e49820880544)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:30.266654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:30.311835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1057/2094] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:30.323328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22262807-e312-4d0f-b882-fb290853325c None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:30.325573 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-22262807-e312-4d0f-b882-fb290853325c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:30.327227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e3bb221e0db476280f7e49820880544'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:30.336350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e3bb221e0db476280f7e49820880544'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:30.337166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e3bb221e0db476280f7e49820880544'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:30.337824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e3bb221e0db476280f7e49820880544'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:30.346681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e3bb221e0db476280f7e49820880544'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:30.372113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1038/2095] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:30 2022] DELETE /identity/v3/users/8e3bb221e0db476280f7e49820880544 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:30.391444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:30.443236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22262807-e312-4d0f-b882-fb290853325c None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:30.525141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce54ed-ed13-4dbf-8fda-addc8cb1fcf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:30.525864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce54ed-ed13-4dbf-8fda-addc8cb1fcf5 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:30.526259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce54ed-ed13-4dbf-8fda-addc8cb1fcf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:30.526508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce54ed-ed13-4dbf-8fda-addc8cb1fcf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:30.528428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5fce54ed-ed13-4dbf-8fda-addc8cb1fcf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:30.647954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce54ed-ed13-4dbf-8fda-addc8cb1fcf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:30.649473 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce54ed-ed13-4dbf-8fda-addc8cb1fcf5 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:30.673487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1058/2096] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:30.758903 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:30.760394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:30.797721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:30.918922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7e322-dfbd-4624-bb36-ef02f5d8a1e9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:30.927025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c88be-69ff-4386-aee5-0a560f01ac1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:30.928107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c88be-69ff-4386-aee5-0a560f01ac1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:30.928550 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c88be-69ff-4386-aee5-0a560f01ac1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:30.928904 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c88be-69ff-4386-aee5-0a560f01ac1d admin admin] PATH_INFO: `/v3/projects/04ed5108c2214df48040694ab5ad3571` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:30.930485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c88be-69ff-4386-aee5-0a560f01ac1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=04ed5108c2214df48040694ab5ad3571)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:30.933864 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c88be-69ff-4386-aee5-0a560f01ac1d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:31.018306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6e0c88be-69ff-4386-aee5-0a560f01ac1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04ed5108c2214df48040694ab5ad3571', 'request_id': 'req-6e0c88be-69ff-4386-aee5-0a560f01ac1d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:31.041796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6e0c88be-69ff-4386-aee5-0a560f01ac1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04ed5108c2214df48040694ab5ad3571', 'request_id': 'req-6e0c88be-69ff-4386-aee5-0a560f01ac1d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:31.046896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6e0c88be-69ff-4386-aee5-0a560f01ac1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04ed5108c2214df48040694ab5ad3571', 'request_id': 'req-6e0c88be-69ff-4386-aee5-0a560f01ac1d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:31.072720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6e0c88be-69ff-4386-aee5-0a560f01ac1d admin admin] The token cache is being invalidate because project 04ed5108c2214df48040694ab5ad3571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:31.072916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6e0c88be-69ff-4386-aee5-0a560f01ac1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:31.074317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6e0c88be-69ff-4386-aee5-0a560f01ac1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:31.080979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1039/2097] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:30 2022] DELETE /identity/v3/projects/04ed5108c2214df48040694ab5ad3571 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:31.082725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d0c47-3ec8-4dc9-b7bf-17b5ff4249b1 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:31.083226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d0c47-3ec8-4dc9-b7bf-17b5ff4249b1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:31.083330 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d0c47-3ec8-4dc9-b7bf-17b5ff4249b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.112225 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-6b2d0c47-3ec8-4dc9-b7bf-17b5ff4249b1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:31.113589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-6b2d0c47-3ec8-4dc9-b7bf-17b5ff4249b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:31.239266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1059/2098] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:31.250206 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26be02-44fd-4ef9-9941-31c20339cc5d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:31.250638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26be02-44fd-4ef9-9941-31c20339cc5d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:31.250997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26be02-44fd-4ef9-9941-31c20339cc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.276038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7e26be02-44fd-4ef9-9941-31c20339cc5d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:31.277802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7e26be02-44fd-4ef9-9941-31c20339cc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:31.299258 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1040/2099] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:31.303166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d0c47-3ec8-4dc9-b7bf-17b5ff4249b1 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:31.314641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d0c47-3ec8-4dc9-b7bf-17b5ff4249b1 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:31.320658 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cac2f-5e2d-4768-9168-285c16a5ad50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:31.321297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cac2f-5e2d-4768-9168-285c16a5ad50 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:31.321921 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cac2f-5e2d-4768-9168-285c16a5ad50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:31.322200 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cac2f-5e2d-4768-9168-285c16a5ad50 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.323967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cac2f-5e2d-4768-9168-285c16a5ad50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:31.324282 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cac2f-5e2d-4768-9168-285c16a5ad50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:31.327851 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cac2f-5e2d-4768-9168-285c16a5ad50 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:31.345398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1060/2100] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:31.351371 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26be02-44fd-4ef9-9941-31c20339cc5d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:31.364958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26be02-44fd-4ef9-9941-31c20339cc5d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:31.370785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84613d4b-3de4-4b96-b266-275c5059d091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:31.371554 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84613d4b-3de4-4b96-b266-275c5059d091 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:31.371928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84613d4b-3de4-4b96-b266-275c5059d091 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:31.372274 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84613d4b-3de4-4b96-b266-275c5059d091 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.372994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84613d4b-3de4-4b96-b266-275c5059d091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:31.376596 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84613d4b-3de4-4b96-b266-275c5059d091 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:31.417600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1041/2101] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:31 2022] POST /identity/v3/projects => generated 365 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:31.422852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cac2f-5e2d-4768-9168-285c16a5ad50 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:31.437006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cac2f-5e2d-4768-9168-285c16a5ad50 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:31.473134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:31.514502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb366c-ac21-44ea-a385-9deba2f53377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:31.515200 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb366c-ac21-44ea-a385-9deba2f53377 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:31.515998 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb366c-ac21-44ea-a385-9deba2f53377 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:31.516276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb366c-ac21-44ea-a385-9deba2f53377 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.516918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb366c-ac21-44ea-a385-9deba2f53377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:31.518857 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb366c-ac21-44ea-a385-9deba2f53377 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:31.542136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1061/2102] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:31 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:31.549168 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84613d4b-3de4-4b96-b266-275c5059d091 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:31.559791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84613d4b-3de4-4b96-b266-275c5059d091 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:31.564647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89db57-f7cd-47b9-951a-1177117ca8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:31.565232 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89db57-f7cd-47b9-951a-1177117ca8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:31.565508 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89db57-f7cd-47b9-951a-1177117ca8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:31.565838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89db57-f7cd-47b9-951a-1177117ca8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.566485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89db57-f7cd-47b9-951a-1177117ca8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:31.568075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89db57-f7cd-47b9-951a-1177117ca8a6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:31.586604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1042/2103] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:31.589324 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb366c-ac21-44ea-a385-9deba2f53377 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:31.600952 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb366c-ac21-44ea-a385-9deba2f53377 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:31.605994 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db19d4a-1ae0-4874-b67f-3e5021d3634a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:31.606282 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19d4a-1ae0-4874-b67f-3e5021d3634a admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:31.606558 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19d4a-1ae0-4874-b67f-3e5021d3634a admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:31.606827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19d4a-1ae0-4874-b67f-3e5021d3634a admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.607468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db19d4a-1ae0-4874-b67f-3e5021d3634a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:31.609608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db19d4a-1ae0-4874-b67f-3e5021d3634a admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:31.625905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1062/2104] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:31.627985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89db57-f7cd-47b9-951a-1177117ca8a6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:31.639850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89db57-f7cd-47b9-951a-1177117ca8a6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:31.646948 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e239e6-98f5-4220-84e8-6d8bb9b2ee54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:31.647626 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e239e6-98f5-4220-84e8-6d8bb9b2ee54 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:31.647961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e239e6-98f5-4220-84e8-6d8bb9b2ee54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:31.648248 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e239e6-98f5-4220-84e8-6d8bb9b2ee54 admin admin] PATH_INFO: `/v3/projects/def12aa06f254776851ce9680c91997e/users/2833e9e33ad94e5bb8177f9c9786563c/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.668965 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e239e6-98f5-4220-84e8-6d8bb9b2ee54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=def12aa06f254776851ce9680c91997e, user_id=2833e9e33ad94e5bb8177f9c9786563c, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:31.673842 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e239e6-98f5-4220-84e8-6d8bb9b2ee54 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:31.703275 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1043/2105] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:31 2022] PUT /identity/v3/projects/def12aa06f254776851ce9680c91997e/users/2833e9e33ad94e5bb8177f9c9786563c/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:31.716309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7172404-0c64-464d-ab5e-08b154557b42 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:31.716309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7172404-0c64-464d-ab5e-08b154557b42 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:31.716309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7172404-0c64-464d-ab5e-08b154557b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.751831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-d7172404-0c64-464d-ab5e-08b154557b42 None None] MFA Rules not processed for user `2833e9e33ad94e5bb8177f9c9786563c`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:31.753837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d7172404-0c64-464d-ab5e-08b154557b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:31.836041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1063/2106] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:31.840691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e239e6-98f5-4220-84e8-6d8bb9b2ee54 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:31.903961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:31.939733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e239e6-98f5-4220-84e8-6d8bb9b2ee54 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:31.945294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9724453-4917-409d-aec1-1e0e64d5ece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:31.945294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9724453-4917-409d-aec1-1e0e64d5ece4 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:31.945710 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9724453-4917-409d-aec1-1e0e64d5ece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:31.946095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9724453-4917-409d-aec1-1e0e64d5ece4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:31.946821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9724453-4917-409d-aec1-1e0e64d5ece4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:31.948878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9724453-4917-409d-aec1-1e0e64d5ece4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:31.990012 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1044/2107] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:31 2022] POST /identity/v3/projects => generated 365 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:31.991196 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7172404-0c64-464d-ab5e-08b154557b42 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:32.008075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7172404-0c64-464d-ab5e-08b154557b42 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:32.111660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef2b6e-9054-4bf8-89cc-ce4bcf6eaeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:32.112288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef2b6e-9054-4bf8-89cc-ce4bcf6eaeb7 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:32.112288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef2b6e-9054-4bf8-89cc-ce4bcf6eaeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:32.112288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef2b6e-9054-4bf8-89cc-ce4bcf6eaeb7 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:32.112977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef2b6e-9054-4bf8-89cc-ce4bcf6eaeb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:32.115571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef2b6e-9054-4bf8-89cc-ce4bcf6eaeb7 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:32.164536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1064/2108] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:31 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:32.165589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9724453-4917-409d-aec1-1e0e64d5ece4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:32.204683 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9724453-4917-409d-aec1-1e0e64d5ece4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:32.227619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98536e21-8f11-4487-82d4-ce20552f05ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:32.228685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98536e21-8f11-4487-82d4-ce20552f05ac admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:32.229130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98536e21-8f11-4487-82d4-ce20552f05ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:32.229310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98536e21-8f11-4487-82d4-ce20552f05ac admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:32.230150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98536e21-8f11-4487-82d4-ce20552f05ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:32.239373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98536e21-8f11-4487-82d4-ce20552f05ac admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:32.268403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052f6a0-6c75-464a-b912-81c4973d5903 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:32.268854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052f6a0-6c75-464a-b912-81c4973d5903 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:32.269799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052f6a0-6c75-464a-b912-81c4973d5903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:32.289029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1045/2109] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:32.291283 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98536e21-8f11-4487-82d4-ce20552f05ac admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:32.311785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-e052f6a0-6c75-464a-b912-81c4973d5903 None None] MFA Rules not processed for user `e60401b675bb4dff9ebc12b6076fdd3f`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:32.313140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e052f6a0-6c75-464a-b912-81c4973d5903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:32.317526 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98536e21-8f11-4487-82d4-ce20552f05ac admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:32.334808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f57b19-00bd-4fcc-824f-484701d06a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:32.335429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f57b19-00bd-4fcc-824f-484701d06a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:32.335702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f57b19-00bd-4fcc-824f-484701d06a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:32.335980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f57b19-00bd-4fcc-824f-484701d06a27 admin admin] PATH_INFO: `/v3/projects/cfc9833adeab42a380a72acdde94a171/users/97b2c2223a814d0f893c117faa44f0c4/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:32.352053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:32.357665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f57b19-00bd-4fcc-824f-484701d06a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc9833adeab42a380a72acdde94a171, user_id=97b2c2223a814d0f893c117faa44f0c4, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:32.360429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f57b19-00bd-4fcc-824f-484701d06a27 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:32.376915 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1046/2110] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:32 2022] PUT /identity/v3/projects/cfc9833adeab42a380a72acdde94a171/users/97b2c2223a814d0f893c117faa44f0c4/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:32.381008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f57b19-00bd-4fcc-824f-484701d06a27 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:32.419919 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1065/2111] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:32.470044 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f57b19-00bd-4fcc-824f-484701d06a27 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:32.476942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b34e34-09cb-404b-b903-3b05fcf85fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:32.477624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b34e34-09cb-404b-b903-3b05fcf85fcf admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:32.477957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b34e34-09cb-404b-b903-3b05fcf85fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:32.478237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b34e34-09cb-404b-b903-3b05fcf85fcf admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:32.479061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b34e34-09cb-404b-b903-3b05fcf85fcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:32.481566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b34e34-09cb-404b-b903-3b05fcf85fcf admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:32.496014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1047/2112] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:32.501324 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052f6a0-6c75-464a-b912-81c4973d5903 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:32.520004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052f6a0-6c75-464a-b912-81c4973d5903 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:32.526876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31511d-764d-4058-8f25-22ddc2bfec06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:32.527599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31511d-764d-4058-8f25-22ddc2bfec06 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:32.528056 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31511d-764d-4058-8f25-22ddc2bfec06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:32.528388 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31511d-764d-4058-8f25-22ddc2bfec06 admin admin] PATH_INFO: `/v3/projects/cfc9833adeab42a380a72acdde94a171/users/97b2c2223a814d0f893c117faa44f0c4/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:32.532473 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31511d-764d-4058-8f25-22ddc2bfec06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc9833adeab42a380a72acdde94a171, user_id=97b2c2223a814d0f893c117faa44f0c4, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:32.536166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31511d-764d-4058-8f25-22ddc2bfec06 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:32.550524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1066/2113] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:32 2022] PUT /identity/v3/projects/cfc9833adeab42a380a72acdde94a171/users/97b2c2223a814d0f893c117faa44f0c4/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:32.560647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34587aad-5f25-4ba6-9f1d-e7101f877b62 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:32.561011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34587aad-5f25-4ba6-9f1d-e7101f877b62 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:32.561343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34587aad-5f25-4ba6-9f1d-e7101f877b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:32.595097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-34587aad-5f25-4ba6-9f1d-e7101f877b62 None None] MFA Rules not processed for user `97b2c2223a814d0f893c117faa44f0c4`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:32.596432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-34587aad-5f25-4ba6-9f1d-e7101f877b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:32.697980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1048/2114] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:32.710926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31511d-764d-4058-8f25-22ddc2bfec06 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:32.712922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-dc31511d-764d-4058-8f25-22ddc2bfec06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:32.785422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31511d-764d-4058-8f25-22ddc2bfec06 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:32.789641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406aa4c5-e11e-452f-b336-d52e96a1ec4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:32.790552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406aa4c5-e11e-452f-b336-d52e96a1ec4f service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:32.790993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406aa4c5-e11e-452f-b336-d52e96a1ec4f service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:32.791295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406aa4c5-e11e-452f-b336-d52e96a1ec4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:32.807212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406aa4c5-e11e-452f-b336-d52e96a1ec4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:32.808666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406aa4c5-e11e-452f-b336-d52e96a1ec4f service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:32.825171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1067/2115] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:32.938725 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34587aad-5f25-4ba6-9f1d-e7101f877b62 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:32.951191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34587aad-5f25-4ba6-9f1d-e7101f877b62 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:32.960458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41eec5-b8f7-4633-8feb-b6d2f3de1205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:32.961463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41eec5-b8f7-4633-8feb-b6d2f3de1205 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:32.961879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41eec5-b8f7-4633-8feb-b6d2f3de1205 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:32.962257 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41eec5-b8f7-4633-8feb-b6d2f3de1205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:32.975731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:33.079197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41eec5-b8f7-4633-8feb-b6d2f3de1205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:33.081416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41eec5-b8f7-4633-8feb-b6d2f3de1205 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:33.099497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1049/2116] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:36.507270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406aa4c5-e11e-452f-b336-d52e96a1ec4f service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:36.675984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406aa4c5-e11e-452f-b336-d52e96a1ec4f service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:36.681095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:36.682261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:36.682751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:36.683227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] PATH_INFO: `/v3/users/2833e9e33ad94e5bb8177f9c9786563c` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:36.685563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2833e9e33ad94e5bb8177f9c9786563c)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:36.688540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:36.766169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2833e9e33ad94e5bb8177f9c9786563c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:36.781202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2833e9e33ad94e5bb8177f9c9786563c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:36.782345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2833e9e33ad94e5bb8177f9c9786563c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:36.783423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2833e9e33ad94e5bb8177f9c9786563c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:36.793431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2833e9e33ad94e5bb8177f9c9786563c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:36.795473 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:36.825413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1068/2117] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:36 2022] DELETE /identity/v3/users/2833e9e33ad94e5bb8177f9c9786563c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:36.829938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf9b1f-2727-491b-8086-a7db4c5c30c2 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:36.830420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf9b1f-2727-491b-8086-a7db4c5c30c2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:36.830518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf9b1f-2727-491b-8086-a7db4c5c30c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:36.881413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-23bf9b1f-2727-491b-8086-a7db4c5c30c2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:36.904189 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-23bf9b1f-2727-491b-8086-a7db4c5c30c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:37.146356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1050/2118] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:37.203189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:37.204898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:37.341592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9468050-65e1-4202-a24b-ab6d0c4bd312 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:37.353895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2328584-0618-493b-b6bf-78df27e16dbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:37.354370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2328584-0618-493b-b6bf-78df27e16dbb service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:37.354652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2328584-0618-493b-b6bf-78df27e16dbb service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:37.354924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2328584-0618-493b-b6bf-78df27e16dbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:37.375410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2328584-0618-493b-b6bf-78df27e16dbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:37.376970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2328584-0618-493b-b6bf-78df27e16dbb service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:37.391604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1069/2119] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:38.770001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf9b1f-2727-491b-8086-a7db4c5c30c2 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:38.772421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-23bf9b1f-2727-491b-8086-a7db4c5c30c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:38.818233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf9b1f-2727-491b-8086-a7db4c5c30c2 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:38.832429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:38.833098 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:38.833391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:38.833667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] PATH_INFO: `/v3/projects/def12aa06f254776851ce9680c91997e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:38.836758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=def12aa06f254776851ce9680c91997e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:38.839425 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:38.949765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416dee-36e7-4b57-9c36-3ad0c4e70e3c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:38.950242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416dee-36e7-4b57-9c36-3ad0c4e70e3c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:38.950379 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c416dee-36e7-4b57-9c36-3ad0c4e70e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:38.988775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'def12aa06f254776851ce9680c91997e', 'request_id': 'req-76c66955-e44e-42cb-9a0e-cf1b260050e1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:38.997633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4c416dee-36e7-4b57-9c36-3ad0c4e70e3c None None] MFA Rules not processed for user `bed19e9a74be46b5bc836c3cd605133f`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:38.999865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'def12aa06f254776851ce9680c91997e', 'request_id': 'req-76c66955-e44e-42cb-9a0e-cf1b260050e1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.004393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4c416dee-36e7-4b57-9c36-3ad0c4e70e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:39.012561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'def12aa06f254776851ce9680c91997e', 'request_id': 'req-76c66955-e44e-42cb-9a0e-cf1b260050e1'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.034552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] The token cache is being invalidate because project def12aa06f254776851ce9680c91997e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:39.034843 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.037303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.041781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1051/2120] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:38 2022] DELETE /identity/v3/projects/def12aa06f254776851ce9680c91997e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:39.046059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:39.047985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:39.093963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:39.130963 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1070/2121] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:39.135093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c416dee-36e7-4b57-9c36-3ad0c4e70e3c None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:39.136631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4c416dee-36e7-4b57-9c36-3ad0c4e70e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:39.140415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c66955-e44e-42cb-9a0e-cf1b260050e1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:39.147333 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:39.148219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.148697 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.149155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] PATH_INFO: `/v3/users/97b2c2223a814d0f893c117faa44f0c4` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.150917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b2c2223a814d0f893c117faa44f0c4)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:39.153546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:39.158393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c416dee-36e7-4b57-9c36-3ad0c4e70e3c None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:39.165322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b3c9d-40af-497e-910e-732722d86a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:39.165982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b3c9d-40af-497e-910e-732722d86a39 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.166259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b3c9d-40af-497e-910e-732722d86a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.166528 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b3c9d-40af-497e-910e-732722d86a39 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.167161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b3c9d-40af-497e-910e-732722d86a39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:39.169059 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b3c9d-40af-497e-910e-732722d86a39 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:39.217797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1071/2122] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:39 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:39.220699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b3c9d-40af-497e-910e-732722d86a39 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:39.230208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b2c2223a814d0f893c117faa44f0c4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.241527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b2c2223a814d0f893c117faa44f0c4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.242186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b2c2223a814d0f893c117faa44f0c4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.243666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b2c2223a814d0f893c117faa44f0c4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.253572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b2c2223a814d0f893c117faa44f0c4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.279919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1052/2123] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:39 2022] DELETE /identity/v3/users/97b2c2223a814d0f893c117faa44f0c4 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:39.306956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:39.328868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b3c9d-40af-497e-910e-732722d86a39 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:39.434908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f5e1-472b-450a-981d-a152f10bcdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:39.435692 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f5e1-472b-450a-981d-a152f10bcdb6 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.436109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f5e1-472b-450a-981d-a152f10bcdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.436453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f5e1-472b-450a-981d-a152f10bcdb6 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.437173 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f5e1-472b-450a-981d-a152f10bcdb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:39.439415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f5e1-472b-450a-981d-a152f10bcdb6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:39.467775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1072/2124] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:39 2022] POST /identity/v3/users => generated 298 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:39.469435 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:39.471692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:39.497601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544ded6-856a-41a7-94c2-bc368c9909c6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:39.503167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5eb56a-7eaa-4fbe-83dc-fdf663a40978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:39.503870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5eb56a-7eaa-4fbe-83dc-fdf663a40978 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.504219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5eb56a-7eaa-4fbe-83dc-fdf663a40978 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.504560 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5eb56a-7eaa-4fbe-83dc-fdf663a40978 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.505489 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5eb56a-7eaa-4fbe-83dc-fdf663a40978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:39.507614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5eb56a-7eaa-4fbe-83dc-fdf663a40978 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:39.524173 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1053/2125] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:39.536757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f5e1-472b-450a-981d-a152f10bcdb6 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:39.559642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f5e1-472b-450a-981d-a152f10bcdb6 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:39.566081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991f4dd-5b89-4ed5-9f7f-00ad612a6b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:39.566849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991f4dd-5b89-4ed5-9f7f-00ad612a6b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.567190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991f4dd-5b89-4ed5-9f7f-00ad612a6b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.567719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991f4dd-5b89-4ed5-9f7f-00ad612a6b6d admin admin] PATH_INFO: `/v3/projects/444e5e44ba7f48348c36f5e2bf864df0/users/5e67e78bbec9437988872c535aadb415/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.591127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991f4dd-5b89-4ed5-9f7f-00ad612a6b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=444e5e44ba7f48348c36f5e2bf864df0, user_id=5e67e78bbec9437988872c535aadb415, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:39.594742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991f4dd-5b89-4ed5-9f7f-00ad612a6b6d admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:39.613619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1073/2126] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:39 2022] PUT /identity/v3/projects/444e5e44ba7f48348c36f5e2bf864df0/users/5e67e78bbec9437988872c535aadb415/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:39.616251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5eb56a-7eaa-4fbe-83dc-fdf663a40978 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:39.633135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5eb56a-7eaa-4fbe-83dc-fdf663a40978 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:39.678821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991f4dd-5b89-4ed5-9f7f-00ad612a6b6d admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:39.680598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b991f4dd-5b89-4ed5-9f7f-00ad612a6b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:39.714643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92592c5f-79c0-4c1b-815e-633c788fbc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:39.715278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92592c5f-79c0-4c1b-815e-633c788fbc2a admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.716239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92592c5f-79c0-4c1b-815e-633c788fbc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.716239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92592c5f-79c0-4c1b-815e-633c788fbc2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.716980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92592c5f-79c0-4c1b-815e-633c788fbc2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:39.719443 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92592c5f-79c0-4c1b-815e-633c788fbc2a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:39.737586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1054/2127] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:39.741787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92592c5f-79c0-4c1b-815e-633c788fbc2a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:39.752966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92592c5f-79c0-4c1b-815e-633c788fbc2a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:39.757878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f66e2-f2e0-4e62-a8e6-91a7cbdbe7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:39.758538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f66e2-f2e0-4e62-a8e6-91a7cbdbe7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.758820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f66e2-f2e0-4e62-a8e6-91a7cbdbe7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.759096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f66e2-f2e0-4e62-a8e6-91a7cbdbe7d6 admin admin] PATH_INFO: `/v3/projects/444e5e44ba7f48348c36f5e2bf864df0/users/5e67e78bbec9437988872c535aadb415/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.761299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f66e2-f2e0-4e62-a8e6-91a7cbdbe7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444e5e44ba7f48348c36f5e2bf864df0, user_id=5e67e78bbec9437988872c535aadb415, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:39.764004 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f66e2-f2e0-4e62-a8e6-91a7cbdbe7d6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:39.776955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991f4dd-5b89-4ed5-9f7f-00ad612a6b6d admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:39.779552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1055/2128] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:39 2022] PUT /identity/v3/projects/444e5e44ba7f48348c36f5e2bf864df0/users/5e67e78bbec9437988872c535aadb415/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:39.854824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86feae5-314a-4edd-9291-a10027f25448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:39.855513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86feae5-314a-4edd-9291-a10027f25448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:39.855882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86feae5-314a-4edd-9291-a10027f25448 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:39.856145 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86feae5-314a-4edd-9291-a10027f25448 admin admin] PATH_INFO: `/v3/projects/cfc9833adeab42a380a72acdde94a171` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:39.857589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86feae5-314a-4edd-9291-a10027f25448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfc9833adeab42a380a72acdde94a171)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:39.860497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86feae5-314a-4edd-9291-a10027f25448 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:39.883597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:39.962378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b86feae5-314a-4edd-9291-a10027f25448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfc9833adeab42a380a72acdde94a171', 'request_id': 'req-b86feae5-314a-4edd-9291-a10027f25448'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.975402 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b86feae5-314a-4edd-9291-a10027f25448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfc9833adeab42a380a72acdde94a171', 'request_id': 'req-b86feae5-314a-4edd-9291-a10027f25448'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:39.988768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b86feae5-314a-4edd-9291-a10027f25448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfc9833adeab42a380a72acdde94a171', 'request_id': 'req-b86feae5-314a-4edd-9291-a10027f25448'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:40.019217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b86feae5-314a-4edd-9291-a10027f25448 admin admin] The token cache is being invalidate because project cfc9833adeab42a380a72acdde94a171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:40.019521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b86feae5-314a-4edd-9291-a10027f25448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:40.022437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b86feae5-314a-4edd-9291-a10027f25448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:40.025855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1074/2129] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:39 2022] DELETE /identity/v3/projects/cfc9833adeab42a380a72acdde94a171 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:40.031612 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7820384-27a6-4b03-abb0-bb64d0c34716 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.031922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7820384-27a6-4b03-abb0-bb64d0c34716 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.032244 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7820384-27a6-4b03-abb0-bb64d0c34716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.079005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d7820384-27a6-4b03-abb0-bb64d0c34716 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:40.080267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d7820384-27a6-4b03-abb0-bb64d0c34716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:40.129587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:40.213893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1056/2130] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:40.221091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69632b-6e79-457b-9960-381a4af6d4c8 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.221491 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69632b-6e79-457b-9960-381a4af6d4c8 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.221782 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69632b-6e79-457b-9960-381a4af6d4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.246830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-5a69632b-6e79-457b-9960-381a4af6d4c8 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:40.248060 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5a69632b-6e79-457b-9960-381a4af6d4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:40.257776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1075/2131] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:40.260690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7820384-27a6-4b03-abb0-bb64d0c34716 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:40.277511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7820384-27a6-4b03-abb0-bb64d0c34716 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:40.284381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f59a6-9c25-46fa-8f9d-42a96962d247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:40.285593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f59a6-9c25-46fa-8f9d-42a96962d247 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.286141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f59a6-9c25-46fa-8f9d-42a96962d247 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.286696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f59a6-9c25-46fa-8f9d-42a96962d247 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.287917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f59a6-9c25-46fa-8f9d-42a96962d247 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:40.288407 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f59a6-9c25-46fa-8f9d-42a96962d247 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:40.290305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f59a6-9c25-46fa-8f9d-42a96962d247 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:40.311636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1057/2132] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:53:40.312803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69632b-6e79-457b-9960-381a4af6d4c8 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:40.323627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69632b-6e79-457b-9960-381a4af6d4c8 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:40.328719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef8b81-0c06-4fac-9983-4b293714bd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:40.329348 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef8b81-0c06-4fac-9983-4b293714bd36 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.329626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef8b81-0c06-4fac-9983-4b293714bd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.329979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef8b81-0c06-4fac-9983-4b293714bd36 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.330665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef8b81-0c06-4fac-9983-4b293714bd36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:40.332745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef8b81-0c06-4fac-9983-4b293714bd36 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:40.370528 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1076/2133] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:40 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:40.375963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f59a6-9c25-46fa-8f9d-42a96962d247 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:40.386926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f59a6-9c25-46fa-8f9d-42a96962d247 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:40.452396 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aacd03-30f0-467c-aea2-f7098eada484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:40.453017 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aacd03-30f0-467c-aea2-f7098eada484 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.453298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aacd03-30f0-467c-aea2-f7098eada484 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.453568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aacd03-30f0-467c-aea2-f7098eada484 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.454245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aacd03-30f0-467c-aea2-f7098eada484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:40.456175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aacd03-30f0-467c-aea2-f7098eada484 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:40.491857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1058/2134] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:40 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:40.496069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef8b81-0c06-4fac-9983-4b293714bd36 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:40.524072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef8b81-0c06-4fac-9983-4b293714bd36 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:40.536711 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248a1e2-7b20-4ecf-a228-5cf9a3a380e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:40.537355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248a1e2-7b20-4ecf-a228-5cf9a3a380e6 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.537639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248a1e2-7b20-4ecf-a228-5cf9a3a380e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.537975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248a1e2-7b20-4ecf-a228-5cf9a3a380e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.538713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248a1e2-7b20-4ecf-a228-5cf9a3a380e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:40.545609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248a1e2-7b20-4ecf-a228-5cf9a3a380e6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:40.570779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1077/2135] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:40.574643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aacd03-30f0-467c-aea2-f7098eada484 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:40.585162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aacd03-30f0-467c-aea2-f7098eada484 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:40.590639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1a391-b6d9-4cd2-b93d-2868674506cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:40.591267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1a391-b6d9-4cd2-b93d-2868674506cf admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.593505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1a391-b6d9-4cd2-b93d-2868674506cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.593827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1a391-b6d9-4cd2-b93d-2868674506cf admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.594518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1a391-b6d9-4cd2-b93d-2868674506cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:40.596162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1a391-b6d9-4cd2-b93d-2868674506cf admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:40.611013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1059/2136] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:40.612562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248a1e2-7b20-4ecf-a228-5cf9a3a380e6 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:40.624454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248a1e2-7b20-4ecf-a228-5cf9a3a380e6 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:40.634969 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632923d-1d0f-4a83-a84f-6523f27a496c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:40.635886 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632923d-1d0f-4a83-a84f-6523f27a496c admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.636775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632923d-1d0f-4a83-a84f-6523f27a496c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.637075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632923d-1d0f-4a83-a84f-6523f27a496c admin admin] PATH_INFO: `/v3/projects/b73602b390214870a4b7299749364c3f/users/6914290fd1b94108afad2f0fd7964b6a/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.664150 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632923d-1d0f-4a83-a84f-6523f27a496c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73602b390214870a4b7299749364c3f, user_id=6914290fd1b94108afad2f0fd7964b6a, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:40.667057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632923d-1d0f-4a83-a84f-6523f27a496c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:40.684097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1078/2137] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:40 2022] PUT /identity/v3/projects/b73602b390214870a4b7299749364c3f/users/6914290fd1b94108afad2f0fd7964b6a/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:40.695853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2234d-f808-402b-9068-7742f707e7d0 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.696522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2234d-f808-402b-9068-7742f707e7d0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.697043 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2234d-f808-402b-9068-7742f707e7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.741341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c1e2234d-f808-402b-9068-7742f707e7d0 None None] MFA Rules not processed for user `6914290fd1b94108afad2f0fd7964b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:40.756025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c1e2234d-f808-402b-9068-7742f707e7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:40.875366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1060/2138] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:40.877332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632923d-1d0f-4a83-a84f-6523f27a496c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:40.986045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632923d-1d0f-4a83-a84f-6523f27a496c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:40.992624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9158cc1-d11f-490f-8d14-d274eaf4ec4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:40.993356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9158cc1-d11f-490f-8d14-d274eaf4ec4a admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.993669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9158cc1-d11f-490f-8d14-d274eaf4ec4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.994457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9158cc1-d11f-490f-8d14-d274eaf4ec4a admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.995462 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9158cc1-d11f-490f-8d14-d274eaf4ec4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:41.005588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9158cc1-d11f-490f-8d14-d274eaf4ec4a admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:41.070158 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1079/2139] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:40 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:41.070833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2234d-f808-402b-9068-7742f707e7d0 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:41.088089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2234d-f808-402b-9068-7742f707e7d0 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:41.108404 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:41.180497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d639c2f-feb7-44e5-9072-9228ffec9fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:41.181309 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d639c2f-feb7-44e5-9072-9228ffec9fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:41.181861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d639c2f-feb7-44e5-9072-9228ffec9fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:41.182330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d639c2f-feb7-44e5-9072-9228ffec9fc2 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:41.183317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d639c2f-feb7-44e5-9072-9228ffec9fc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:41.187619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d639c2f-feb7-44e5-9072-9228ffec9fc2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:41.253702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1061/2140] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:41 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:41.257691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9158cc1-d11f-490f-8d14-d274eaf4ec4a admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:41.273146 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9158cc1-d11f-490f-8d14-d274eaf4ec4a admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:41.301267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955daa8-2958-4e8e-bb77-64ce0edebd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:41.302192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955daa8-2958-4e8e-bb77-64ce0edebd38 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:41.302571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955daa8-2958-4e8e-bb77-64ce0edebd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:41.302941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4955daa8-2958-4e8e-bb77-64ce0edebd38 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:41.303747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955daa8-2958-4e8e-bb77-64ce0edebd38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:41.307663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4955daa8-2958-4e8e-bb77-64ce0edebd38 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:41.338014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1080/2141] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:41.342654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d639c2f-feb7-44e5-9072-9228ffec9fc2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:41.354215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d639c2f-feb7-44e5-9072-9228ffec9fc2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:41.359777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4349a1-10a9-459b-adcc-e00a28daaaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:41.360644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4349a1-10a9-459b-adcc-e00a28daaaa7 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:41.361107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4349a1-10a9-459b-adcc-e00a28daaaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:41.361669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4349a1-10a9-459b-adcc-e00a28daaaa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:41.362736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4349a1-10a9-459b-adcc-e00a28daaaa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:41.364700 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4349a1-10a9-459b-adcc-e00a28daaaa7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:41.378085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1062/2142] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:53:41.382966 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955daa8-2958-4e8e-bb77-64ce0edebd38 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:41.397905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4955daa8-2958-4e8e-bb77-64ce0edebd38 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:41.411215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f93af-ca30-4a04-a9e5-d9a31efadfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:41.412941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f93af-ca30-4a04-a9e5-d9a31efadfec admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:41.417587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f93af-ca30-4a04-a9e5-d9a31efadfec admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:41.418092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f93af-ca30-4a04-a9e5-d9a31efadfec admin admin] PATH_INFO: `/v3/projects/ac6e56930727476eb05dc5956a169895/users/3d4553a00c8f49e18ab8a3963367cdd3/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:41.435012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:41.462506 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f93af-ca30-4a04-a9e5-d9a31efadfec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac6e56930727476eb05dc5956a169895, user_id=3d4553a00c8f49e18ab8a3963367cdd3, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:41.468259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f93af-ca30-4a04-a9e5-d9a31efadfec admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:41.495190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1081/2143] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:41 2022] PUT /identity/v3/projects/ac6e56930727476eb05dc5956a169895/users/3d4553a00c8f49e18ab8a3963367cdd3/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:41.506739 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84def-38dd-4fdc-8b4d-4316206dc410 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:41.507151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84def-38dd-4fdc-8b4d-4316206dc410 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:41.507530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84def-38dd-4fdc-8b4d-4316206dc410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:41.579060 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1ed84def-38dd-4fdc-8b4d-4316206dc410 None None] MFA Rules not processed for user `3d4553a00c8f49e18ab8a3963367cdd3`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:41.581405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1ed84def-38dd-4fdc-8b4d-4316206dc410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:41.752687 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1063/2144] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:41.758630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f93af-ca30-4a04-a9e5-d9a31efadfec admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:41.923787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f93af-ca30-4a04-a9e5-d9a31efadfec admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:41.939157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f691ec8-ced2-4df6-af0f-8375e07f3aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:41.939795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f691ec8-ced2-4df6-af0f-8375e07f3aab admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:41.940282 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f691ec8-ced2-4df6-af0f-8375e07f3aab admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:41.940567 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f691ec8-ced2-4df6-af0f-8375e07f3aab admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:41.941227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f691ec8-ced2-4df6-af0f-8375e07f3aab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:41.944954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f691ec8-ced2-4df6-af0f-8375e07f3aab admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:42.010405 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1082/2145] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:41 2022] POST /identity/v3/projects => generated 383 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:42.014064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed84def-38dd-4fdc-8b4d-4316206dc410 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:42.034683 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed84def-38dd-4fdc-8b4d-4316206dc410 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:42.151415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ddb40-6308-40ae-81f5-9d91050d28a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:42.152412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ddb40-6308-40ae-81f5-9d91050d28a6 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:42.152902 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ddb40-6308-40ae-81f5-9d91050d28a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:42.153382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ddb40-6308-40ae-81f5-9d91050d28a6 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:42.161714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ddb40-6308-40ae-81f5-9d91050d28a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:42.164886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ddb40-6308-40ae-81f5-9d91050d28a6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:42.196178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1064/2146] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:42 2022] POST /identity/v3/users => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:42.202647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f691ec8-ced2-4df6-af0f-8375e07f3aab admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:42.234757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f691ec8-ced2-4df6-af0f-8375e07f3aab admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:42.244772 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6be582-d850-44a9-a37b-36954e0e4608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:42.246149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6be582-d850-44a9-a37b-36954e0e4608 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:42.246697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6be582-d850-44a9-a37b-36954e0e4608 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:42.247239 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6be582-d850-44a9-a37b-36954e0e4608 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:42.249077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6be582-d850-44a9-a37b-36954e0e4608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:42.252421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6be582-d850-44a9-a37b-36954e0e4608 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:42.284246 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ddb40-6308-40ae-81f5-9d91050d28a6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:42.285159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1083/2147] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:42.297150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ddb40-6308-40ae-81f5-9d91050d28a6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:42.332559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535be31-3151-4edb-b790-ed971587e47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:42.333265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535be31-3151-4edb-b790-ed971587e47f admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:42.333544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535be31-3151-4edb-b790-ed971587e47f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:42.333843 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535be31-3151-4edb-b790-ed971587e47f admin admin] PATH_INFO: `/v3/projects/74466c8b96a84bffbe9e0d27dc286b8b/users/11b404c9783342bb95e88dd903ffe255/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:42.389546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535be31-3151-4edb-b790-ed971587e47f admin admin] RBAC: Authorizing `identity:create_grant(project_id=74466c8b96a84bffbe9e0d27dc286b8b, user_id=11b404c9783342bb95e88dd903ffe255, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:42.389546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535be31-3151-4edb-b790-ed971587e47f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:42.389546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:42.420767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1065/2148] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:42 2022] PUT /identity/v3/projects/74466c8b96a84bffbe9e0d27dc286b8b/users/11b404c9783342bb95e88dd903ffe255/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:42.432973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6be582-d850-44a9-a37b-36954e0e4608 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:42.448113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6be582-d850-44a9-a37b-36954e0e4608 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:42.584581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7b5bf-d3f2-4e55-942e-8f8704473601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:42.585224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7b5bf-d3f2-4e55-942e-8f8704473601 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:42.585513 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7b5bf-d3f2-4e55-942e-8f8704473601 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:42.585791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7b5bf-d3f2-4e55-942e-8f8704473601 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:42.586720 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7b5bf-d3f2-4e55-942e-8f8704473601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:42.588654 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7b5bf-d3f2-4e55-942e-8f8704473601 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:42.613035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1084/2149] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:42.617187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535be31-3151-4edb-b790-ed971587e47f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:42.630800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535be31-3151-4edb-b790-ed971587e47f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:42.641003 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530aac8-510c-49db-a223-67269eff6489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:42.642077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530aac8-510c-49db-a223-67269eff6489 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:42.642549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530aac8-510c-49db-a223-67269eff6489 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:42.642996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530aac8-510c-49db-a223-67269eff6489 admin admin] PATH_INFO: `/v3/projects/74466c8b96a84bffbe9e0d27dc286b8b/users/11b404c9783342bb95e88dd903ffe255/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:42.646650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530aac8-510c-49db-a223-67269eff6489 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74466c8b96a84bffbe9e0d27dc286b8b, user_id=11b404c9783342bb95e88dd903ffe255, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:42.651432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530aac8-510c-49db-a223-67269eff6489 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:42.718057 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1066/2150] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:42 2022] PUT /identity/v3/projects/74466c8b96a84bffbe9e0d27dc286b8b/users/11b404c9783342bb95e88dd903ffe255/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:42.729540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b4b27-638b-4dcc-9f04-6a3a57d17ae0 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:42.729852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b4b27-638b-4dcc-9f04-6a3a57d17ae0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:42.730137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b4b27-638b-4dcc-9f04-6a3a57d17ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:42.797448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-998b4b27-638b-4dcc-9f04-6a3a57d17ae0 None None] MFA Rules not processed for user `11b404c9783342bb95e88dd903ffe255`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:42.803046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-998b4b27-638b-4dcc-9f04-6a3a57d17ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:42.950093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:43.025447 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1085/2151] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:43.037329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530aac8-510c-49db-a223-67269eff6489 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:43.039719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f530aac8-510c-49db-a223-67269eff6489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:43.218799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530aac8-510c-49db-a223-67269eff6489 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:43.228725 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b12ac-3a13-44b6-8f31-775e44e2a6b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:43.231254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b12ac-3a13-44b6-8f31-775e44e2a6b7 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:43.233337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b12ac-3a13-44b6-8f31-775e44e2a6b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:43.248462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b12ac-3a13-44b6-8f31-775e44e2a6b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:43.274714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b12ac-3a13-44b6-8f31-775e44e2a6b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:43.289318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b12ac-3a13-44b6-8f31-775e44e2a6b7 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:43.307937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1067/2152] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:44.109179 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b4b27-638b-4dcc-9f04-6a3a57d17ae0 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:44.111299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-998b4b27-638b-4dcc-9f04-6a3a57d17ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:44.139275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b4b27-638b-4dcc-9f04-6a3a57d17ae0 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:44.143259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eea188-21aa-49a1-8d86-395956893aa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:44.143904 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eea188-21aa-49a1-8d86-395956893aa4 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:44.144203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eea188-21aa-49a1-8d86-395956893aa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:44.144474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eea188-21aa-49a1-8d86-395956893aa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:44.283051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eea188-21aa-49a1-8d86-395956893aa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:44.284227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eea188-21aa-49a1-8d86-395956893aa4 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:44.315425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1086/2153] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:45.976220 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b86e5-f266-4de9-93b3-e464facc3ef5 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:45.976679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b86e5-f266-4de9-93b3-e464facc3ef5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:45.976759 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b86e5-f266-4de9-93b3-e464facc3ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.013084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-142b86e5-f266-4de9-93b3-e464facc3ef5 None None] MFA Rules not processed for user `5e67e78bbec9437988872c535aadb415`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:46.014382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-142b86e5-f266-4de9-93b3-e464facc3ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.136997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1068/2154] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:46.152327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eea188-21aa-49a1-8d86-395956893aa4 service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:46.154409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-54eea188-21aa-49a1-8d86-395956893aa4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.247047 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eea188-21aa-49a1-8d86-395956893aa4 service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:46.251551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006275d-ae5a-41ab-b770-992c634aae1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e27f5b81c704dbebbd6a83b2713e237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:46.252176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006275d-ae5a-41ab-b770-992c634aae1c service glance] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.252482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006275d-ae5a-41ab-b770-992c634aae1c service glance] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.252756 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006275d-ae5a-41ab-b770-992c634aae1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.254508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4006275d-ae5a-41ab-b770-992c634aae1c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.355762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006275d-ae5a-41ab-b770-992c634aae1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:46.358657 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006275d-ae5a-41ab-b770-992c634aae1c service glance] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:46.371218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1087/2155] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:46.568769 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ced22-1a58-49d3-956b-7aba7eab6d98 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.569084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ced22-1a58-49d3-956b-7aba7eab6d98 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.569362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ced22-1a58-49d3-956b-7aba7eab6d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.598282 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-be0ced22-1a58-49d3-956b-7aba7eab6d98 None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:46.599412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-be0ced22-1a58-49d3-956b-7aba7eab6d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.666651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1069/2156] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 13:53:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:46.680241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006275d-ae5a-41ab-b770-992c634aae1c service glance] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:46.681721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4006275d-ae5a-41ab-b770-992c634aae1c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.728506 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:46.767193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006275d-ae5a-41ab-b770-992c634aae1c service glance] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:46.773994 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516de76b-bf3b-466c-b969-337a698ecc99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:46.774632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516de76b-bf3b-466c-b969-337a698ecc99 service swift] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.774912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516de76b-bf3b-466c-b969-337a698ecc99 service swift] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.775183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516de76b-bf3b-466c-b969-337a698ecc99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.785105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516de76b-bf3b-466c-b969-337a698ecc99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:46.787058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516de76b-bf3b-466c-b969-337a698ecc99 service swift] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:46.797993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1088/2157] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 13:53:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:53:49.325678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ced22-1a58-49d3-956b-7aba7eab6d98 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:49.337325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ced22-1a58-49d3-956b-7aba7eab6d98 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:49.341241 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185583a9-32bc-4888-97bb-09fa2c082b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:49.350340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185583a9-32bc-4888-97bb-09fa2c082b29 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:49.350698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185583a9-32bc-4888-97bb-09fa2c082b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:49.350977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185583a9-32bc-4888-97bb-09fa2c082b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:49.373804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:49.465928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185583a9-32bc-4888-97bb-09fa2c082b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:49.468380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185583a9-32bc-4888-97bb-09fa2c082b29 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:49.494693 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1070/2158] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:52.531505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516de76b-bf3b-466c-b969-337a698ecc99 service swift] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:52.662395 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516de76b-bf3b-466c-b969-337a698ecc99 service swift] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:52.667584 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:52.668228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:52.668498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:52.668913 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] PATH_INFO: `/v3/users/6914290fd1b94108afad2f0fd7964b6a` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:52.669967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=6914290fd1b94108afad2f0fd7964b6a)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:52.684669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:52.761109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6914290fd1b94108afad2f0fd7964b6a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:52.774116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6914290fd1b94108afad2f0fd7964b6a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:52.775095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6914290fd1b94108afad2f0fd7964b6a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:52.775816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6914290fd1b94108afad2f0fd7964b6a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:52.789763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6914290fd1b94108afad2f0fd7964b6a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:52.808421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1089/2159] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:52 2022] DELETE /identity/v3/users/6914290fd1b94108afad2f0fd7964b6a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:52.813239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad19e08-11ab-4b9e-88db-b64354d259d7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:52.813785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad19e08-11ab-4b9e-88db-b64354d259d7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:52.813877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad19e08-11ab-4b9e-88db-b64354d259d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:52.835596 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-aad19e08-11ab-4b9e-88db-b64354d259d7 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:52.837907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-aad19e08-11ab-4b9e-88db-b64354d259d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:52.927806 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1071/2160] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:52.946996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:52.948914 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:53.023389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:53.040799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c6419b-4a27-4989-94e3-9b8d30dd46ed admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:53.048520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abcc0c-63bb-44f3-8af0-0d22546d8878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:53.049432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abcc0c-63bb-44f3-8af0-0d22546d8878 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:53.050084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abcc0c-63bb-44f3-8af0-0d22546d8878 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:53.050523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abcc0c-63bb-44f3-8af0-0d22546d8878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:53.063489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abcc0c-63bb-44f3-8af0-0d22546d8878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:53.066828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abcc0c-63bb-44f3-8af0-0d22546d8878 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:53.079747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1090/2161] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:53.579655 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad19e08-11ab-4b9e-88db-b64354d259d7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:53.581354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-aad19e08-11ab-4b9e-88db-b64354d259d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:53.622373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad19e08-11ab-4b9e-88db-b64354d259d7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:53.635752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:53.636633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:53.637556 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:53.638017 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] PATH_INFO: `/v3/projects/b73602b390214870a4b7299749364c3f` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:53.640183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b73602b390214870a4b7299749364c3f)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:53.642302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:53.708898 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b73602b390214870a4b7299749364c3f', 'request_id': 'req-08591e17-f82f-479e-a985-b9afeffc3e9a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:53.722146 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b73602b390214870a4b7299749364c3f', 'request_id': 'req-08591e17-f82f-479e-a985-b9afeffc3e9a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:53.730169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b73602b390214870a4b7299749364c3f', 'request_id': 'req-08591e17-f82f-479e-a985-b9afeffc3e9a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:53.756906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] The token cache is being invalidate because project b73602b390214870a4b7299749364c3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:53.757482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:53.758267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:53.760832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1072/2162] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:53 2022] DELETE /identity/v3/projects/b73602b390214870a4b7299749364c3f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:53.765109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abcc0c-63bb-44f3-8af0-0d22546d8878 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:53.776521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abcc0c-63bb-44f3-8af0-0d22546d8878 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:53.873394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:53.874008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:53.874286 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:53.874691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] PATH_INFO: `/v3/users/3d4553a00c8f49e18ab8a3963367cdd3` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:53.875843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d4553a00c8f49e18ab8a3963367cdd3)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:53.878634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:53.997884 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d4553a00c8f49e18ab8a3963367cdd3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.014337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d4553a00c8f49e18ab8a3963367cdd3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.015235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d4553a00c8f49e18ab8a3963367cdd3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.016166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d4553a00c8f49e18ab8a3963367cdd3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.032784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d4553a00c8f49e18ab8a3963367cdd3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.068823 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1091/2163] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:53 2022] DELETE /identity/v3/users/3d4553a00c8f49e18ab8a3963367cdd3 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:54.434796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:54.437025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:54.469684 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08591e17-f82f-479e-a985-b9afeffc3e9a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:54.539742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:54.577465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:54.578320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:54.578701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:54.579073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] PATH_INFO: `/v3/projects/ac6e56930727476eb05dc5956a169895` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:54.580311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac6e56930727476eb05dc5956a169895)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:54.582481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:54.651562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac6e56930727476eb05dc5956a169895', 'request_id': 'req-97d0e01e-991f-4fa8-be11-b9d42b24f99f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.659752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac6e56930727476eb05dc5956a169895', 'request_id': 'req-97d0e01e-991f-4fa8-be11-b9d42b24f99f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.667200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac6e56930727476eb05dc5956a169895', 'request_id': 'req-97d0e01e-991f-4fa8-be11-b9d42b24f99f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.686822 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] The token cache is being invalidate because project ac6e56930727476eb05dc5956a169895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:54.687433 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.688745 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.690734 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1073/2164] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:54 2022] DELETE /identity/v3/projects/ac6e56930727476eb05dc5956a169895 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:54.694437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:54.696571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:54.716848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e834e7f-ca2c-4f0c-af16-aef896b1f431 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:54.835219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4fda5-3be4-4e2a-8f45-49c68cb92e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:54.836869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4fda5-3be4-4e2a-8f45-49c68cb92e8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:54.837308 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4fda5-3be4-4e2a-8f45-49c68cb92e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:54.837751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4fda5-3be4-4e2a-8f45-49c68cb92e8b admin admin] PATH_INFO: `/v3/users/11b404c9783342bb95e88dd903ffe255` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:54.843839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4fda5-3be4-4e2a-8f45-49c68cb92e8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b404c9783342bb95e88dd903ffe255)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:54.846131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc4fda5-3be4-4e2a-8f45-49c68cb92e8b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:54.853835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:54.932645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5dc4fda5-3be4-4e2a-8f45-49c68cb92e8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b404c9783342bb95e88dd903ffe255'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.961777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5dc4fda5-3be4-4e2a-8f45-49c68cb92e8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b404c9783342bb95e88dd903ffe255'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.962660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5dc4fda5-3be4-4e2a-8f45-49c68cb92e8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b404c9783342bb95e88dd903ffe255'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.976200 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5dc4fda5-3be4-4e2a-8f45-49c68cb92e8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b404c9783342bb95e88dd903ffe255'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:54.998299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5dc4fda5-3be4-4e2a-8f45-49c68cb92e8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b404c9783342bb95e88dd903ffe255'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:55.054829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1092/2165] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:54 2022] DELETE /identity/v3/users/11b404c9783342bb95e88dd903ffe255 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:53:55.678209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:55.680323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:55.704138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0e01e-991f-4fa8-be11-b9d42b24f99f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:55.838453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:55.839489 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:55.840016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:55.840500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f admin admin] PATH_INFO: `/v3/projects/74466c8b96a84bffbe9e0d27dc286b8b` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:55.843120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f admin admin] RBAC: Authorizing `identity:delete_project(project_id=74466c8b96a84bffbe9e0d27dc286b8b)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:55.845477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:55.934911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74466c8b96a84bffbe9e0d27dc286b8b', 'request_id': 'req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:55.944586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74466c8b96a84bffbe9e0d27dc286b8b', 'request_id': 'req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:55.955192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74466c8b96a84bffbe9e0d27dc286b8b', 'request_id': 'req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:55.960180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:55.977765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f admin admin] The token cache is being invalidate because project 74466c8b96a84bffbe9e0d27dc286b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:53:55.978238 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:55.979555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ad4ffd5f-da42-42f5-a89e-441b84a0a11f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:55.981776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1074/2166] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:53:55 2022] DELETE /identity/v3/projects/74466c8b96a84bffbe9e0d27dc286b8b => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:55.987987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139545b8-5a70-4c2d-9363-bc277de5ef34 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:55.988474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139545b8-5a70-4c2d-9363-bc277de5ef34 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:55.988577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139545b8-5a70-4c2d-9363-bc277de5ef34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.019905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-139545b8-5a70-4c2d-9363-bc277de5ef34 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:56.021130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-139545b8-5a70-4c2d-9363-bc277de5ef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:56.124467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1093/2167] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:56.132488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618019ba-d34b-4029-97fa-6c0e8b5f8b7d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.132794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618019ba-d34b-4029-97fa-6c0e8b5f8b7d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.133038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618019ba-d34b-4029-97fa-6c0e8b5f8b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.161457 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-618019ba-d34b-4029-97fa-6c0e8b5f8b7d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:56.162713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-618019ba-d34b-4029-97fa-6c0e8b5f8b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:56.171235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1075/2168] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:56.175349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139545b8-5a70-4c2d-9363-bc277de5ef34 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:56.186908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139545b8-5a70-4c2d-9363-bc277de5ef34 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:56.192130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42edf44-0521-4259-bad2-a7fe023f1ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:56.192991 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42edf44-0521-4259-bad2-a7fe023f1ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.193444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42edf44-0521-4259-bad2-a7fe023f1ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.193938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42edf44-0521-4259-bad2-a7fe023f1ba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.194791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42edf44-0521-4259-bad2-a7fe023f1ba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:53:56.195288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42edf44-0521-4259-bad2-a7fe023f1ba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:56.197097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42edf44-0521-4259-bad2-a7fe023f1ba5 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:56.210109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1094/2169] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:53:56.214688 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618019ba-d34b-4029-97fa-6c0e8b5f8b7d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:56.224977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618019ba-d34b-4029-97fa-6c0e8b5f8b7d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:56.230411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6234b-f7b2-4a0a-bb16-d70915dc239f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:56.231319 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6234b-f7b2-4a0a-bb16-d70915dc239f admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.231803 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6234b-f7b2-4a0a-bb16-d70915dc239f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.232281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6234b-f7b2-4a0a-bb16-d70915dc239f admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.233179 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6234b-f7b2-4a0a-bb16-d70915dc239f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:56.235669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6234b-f7b2-4a0a-bb16-d70915dc239f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:56.286029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1076/2170] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:56 2022] POST /identity/v3/projects => generated 399 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:56.291629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42edf44-0521-4259-bad2-a7fe023f1ba5 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:56.302818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42edf44-0521-4259-bad2-a7fe023f1ba5 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:56.381170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c00612-f57b-4e79-b5c9-6c57ed60c36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:56.382137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c00612-f57b-4e79-b5c9-6c57ed60c36f admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.382599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c00612-f57b-4e79-b5c9-6c57ed60c36f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.383025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c00612-f57b-4e79-b5c9-6c57ed60c36f admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.383823 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c00612-f57b-4e79-b5c9-6c57ed60c36f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:56.385764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c00612-f57b-4e79-b5c9-6c57ed60c36f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:56.411198 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1095/2171] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:56 2022] POST /identity/v3/users => generated 300 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:56.415039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6234b-f7b2-4a0a-bb16-d70915dc239f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:56.428339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6234b-f7b2-4a0a-bb16-d70915dc239f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:56.438740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5eaa8-a2a6-4cd2-ad39-340b35f17286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:56.441297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5eaa8-a2a6-4cd2-ad39-340b35f17286 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.441585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5eaa8-a2a6-4cd2-ad39-340b35f17286 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.441895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5eaa8-a2a6-4cd2-ad39-340b35f17286 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.442561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5eaa8-a2a6-4cd2-ad39-340b35f17286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:56.444252 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5eaa8-a2a6-4cd2-ad39-340b35f17286 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:56.469232 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1077/2172] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:56.469561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c00612-f57b-4e79-b5c9-6c57ed60c36f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:56.475283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:56.481447 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c00612-f57b-4e79-b5c9-6c57ed60c36f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:56.496175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df81c7-1e4a-45d6-ad94-4c6bc56e6a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:56.496989 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df81c7-1e4a-45d6-ad94-4c6bc56e6a7f admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.497356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df81c7-1e4a-45d6-ad94-4c6bc56e6a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.497691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df81c7-1e4a-45d6-ad94-4c6bc56e6a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.498473 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df81c7-1e4a-45d6-ad94-4c6bc56e6a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:56.500213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df81c7-1e4a-45d6-ad94-4c6bc56e6a7f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:56.522136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1096/2173] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:56.524007 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5eaa8-a2a6-4cd2-ad39-340b35f17286 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:56.548553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5eaa8-a2a6-4cd2-ad39-340b35f17286 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:56.557126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20282cc-f8b6-4e1b-b842-3bc5ad7a38fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:56.558338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20282cc-f8b6-4e1b-b842-3bc5ad7a38fa admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.558701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20282cc-f8b6-4e1b-b842-3bc5ad7a38fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.559052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20282cc-f8b6-4e1b-b842-3bc5ad7a38fa admin admin] PATH_INFO: `/v3/projects/de1abbe94e2348c4831f2efd35fec25e/users/8b445ff1440541198b5d48aaab3b6a63/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.589614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20282cc-f8b6-4e1b-b842-3bc5ad7a38fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1abbe94e2348c4831f2efd35fec25e, user_id=8b445ff1440541198b5d48aaab3b6a63, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:56.593029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20282cc-f8b6-4e1b-b842-3bc5ad7a38fa admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:56.623991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1078/2174] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:56 2022] PUT /identity/v3/projects/de1abbe94e2348c4831f2efd35fec25e/users/8b445ff1440541198b5d48aaab3b6a63/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:56.626553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f1439-5d1d-4a00-bc7b-61dbc64f85c4 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.638604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f1439-5d1d-4a00-bc7b-61dbc64f85c4 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.638700 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f1439-5d1d-4a00-bc7b-61dbc64f85c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.673317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-5d2f1439-5d1d-4a00-bc7b-61dbc64f85c4 None None] MFA Rules not processed for user `8b445ff1440541198b5d48aaab3b6a63`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:56.674803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5d2f1439-5d1d-4a00-bc7b-61dbc64f85c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:56.809023 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1097/2175] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:56 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:56.813554 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20282cc-f8b6-4e1b-b842-3bc5ad7a38fa admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:56.931186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20282cc-f8b6-4e1b-b842-3bc5ad7a38fa admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:56.941873 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951e61b-991b-4b71-8935-2b53cbc2fc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:56.942634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951e61b-991b-4b71-8935-2b53cbc2fc49 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:56.942981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951e61b-991b-4b71-8935-2b53cbc2fc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:56.943449 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951e61b-991b-4b71-8935-2b53cbc2fc49 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:56.944192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951e61b-991b-4b71-8935-2b53cbc2fc49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:56.946228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951e61b-991b-4b71-8935-2b53cbc2fc49 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:57.019567 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f1439-5d1d-4a00-bc7b-61dbc64f85c4 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:57.027523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1079/2176] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:53:56 2022] POST /identity/v3/projects => generated 399 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:53:57.033661 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f1439-5d1d-4a00-bc7b-61dbc64f85c4 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:57.210400 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514f1d8-091b-45a4-b471-edf32cea5853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:57.211979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514f1d8-091b-45a4-b471-edf32cea5853 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:57.212439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514f1d8-091b-45a4-b471-edf32cea5853 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:57.212875 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514f1d8-091b-45a4-b471-edf32cea5853 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:57.213696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514f1d8-091b-45a4-b471-edf32cea5853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:57.216606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514f1d8-091b-45a4-b471-edf32cea5853 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:57.245116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1098/2177] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:53:57 2022] POST /identity/v3/users => generated 306 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:53:57.249516 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951e61b-991b-4b71-8935-2b53cbc2fc49 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:57.268202 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951e61b-991b-4b71-8935-2b53cbc2fc49 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:57.275482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20671f9-942d-41d1-a1f9-22d72e49df15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:57.277727 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20671f9-942d-41d1-a1f9-22d72e49df15 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:57.278321 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20671f9-942d-41d1-a1f9-22d72e49df15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:57.278796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20671f9-942d-41d1-a1f9-22d72e49df15 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:57.279806 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20671f9-942d-41d1-a1f9-22d72e49df15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:57.282072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20671f9-942d-41d1-a1f9-22d72e49df15 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:57.303929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1080/2178] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:57.308537 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514f1d8-091b-45a4-b471-edf32cea5853 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:57.322248 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514f1d8-091b-45a4-b471-edf32cea5853 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:57.327721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d629f-f92e-4578-a317-e42566afe891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:57.328396 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d629f-f92e-4578-a317-e42566afe891 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:57.328677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d629f-f92e-4578-a317-e42566afe891 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:57.328956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d629f-f92e-4578-a317-e42566afe891 admin admin] PATH_INFO: `/v3/projects/e630732383224c92aa1850f04276546b/users/21451dfe16554b24ac91aaeb9439d793/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:57.369961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d629f-f92e-4578-a317-e42566afe891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e630732383224c92aa1850f04276546b, user_id=21451dfe16554b24ac91aaeb9439d793, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:57.378498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d629f-f92e-4578-a317-e42566afe891 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:57.408003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1099/2179] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:57 2022] PUT /identity/v3/projects/e630732383224c92aa1850f04276546b/users/21451dfe16554b24ac91aaeb9439d793/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:57.412014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20671f9-942d-41d1-a1f9-22d72e49df15 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:57.428012 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20671f9-942d-41d1-a1f9-22d72e49df15 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:57.545398 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3b9be-5ee8-422d-9606-9bd32be7aa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:57.546819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3b9be-5ee8-422d-9606-9bd32be7aa09 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:57.547318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3b9be-5ee8-422d-9606-9bd32be7aa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:57.547853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3b9be-5ee8-422d-9606-9bd32be7aa09 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:57.548733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3b9be-5ee8-422d-9606-9bd32be7aa09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:57.550606 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3b9be-5ee8-422d-9606-9bd32be7aa09 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:57.563993 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1081/2180] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:53:57.568165 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d629f-f92e-4578-a317-e42566afe891 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:57.581358 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d629f-f92e-4578-a317-e42566afe891 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:57.587280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddf2c8-a5bf-43e3-bda5-231ec679356f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:57.588332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddf2c8-a5bf-43e3-bda5-231ec679356f admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:57.588822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddf2c8-a5bf-43e3-bda5-231ec679356f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:57.589301 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddf2c8-a5bf-43e3-bda5-231ec679356f admin admin] PATH_INFO: `/v3/projects/e630732383224c92aa1850f04276546b/users/21451dfe16554b24ac91aaeb9439d793/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:57.591653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddf2c8-a5bf-43e3-bda5-231ec679356f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e630732383224c92aa1850f04276546b, user_id=21451dfe16554b24ac91aaeb9439d793, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:57.595283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddf2c8-a5bf-43e3-bda5-231ec679356f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:57.619894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1100/2181] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:53:57 2022] PUT /identity/v3/projects/e630732383224c92aa1850f04276546b/users/21451dfe16554b24ac91aaeb9439d793/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:53:57.628881 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7090bb-7521-47f9-8678-bc543ad7da21 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:57.629156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7090bb-7521-47f9-8678-bc543ad7da21 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:57.629426 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7090bb-7521-47f9-8678-bc543ad7da21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:57.643952 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:57.705264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ad7090bb-7521-47f9-8678-bc543ad7da21 None None] MFA Rules not processed for user `21451dfe16554b24ac91aaeb9439d793`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:57.706586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ad7090bb-7521-47f9-8678-bc543ad7da21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:57.881775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1082/2182] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:53:57 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:57.914755 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddf2c8-a5bf-43e3-bda5-231ec679356f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:57.917787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-aeddf2c8-a5bf-43e3-bda5-231ec679356f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:57.984777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:58.032135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddf2c8-a5bf-43e3-bda5-231ec679356f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:58.036379 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68a6c7-26ab-4e92-9d01-389a96e8c209 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:58.037084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68a6c7-26ab-4e92-9d01-389a96e8c209 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:58.038051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68a6c7-26ab-4e92-9d01-389a96e8c209 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:58.041832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68a6c7-26ab-4e92-9d01-389a96e8c209 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:58.055214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68a6c7-26ab-4e92-9d01-389a96e8c209 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:58.056726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68a6c7-26ab-4e92-9d01-389a96e8c209 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:58.072588 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1101/2183] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:57 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:58.195286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7090bb-7521-47f9-8678-bc543ad7da21 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:58.209857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7090bb-7521-47f9-8678-bc543ad7da21 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:58.214815 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5484d7b-fe13-48e7-9f74-e30d668e2193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:58.215572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5484d7b-fe13-48e7-9f74-e30d668e2193 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:58.215955 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5484d7b-fe13-48e7-9f74-e30d668e2193 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:58.216321 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5484d7b-fe13-48e7-9f74-e30d668e2193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:58.290703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5484d7b-fe13-48e7-9f74-e30d668e2193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:58.292396 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5484d7b-fe13-48e7-9f74-e30d668e2193 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:58.305644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1083/2184] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:58 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:59.611988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d7a3d-78c2-4121-970c-19737038cca1 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:59.612446 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d7a3d-78c2-4121-970c-19737038cca1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:59.612545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d7a3d-78c2-4121-970c-19737038cca1 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:59.614132 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1102/2185] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 13:53:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:59.655462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314ba8d-018f-45bd-853b-78347634f978 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:59.656047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314ba8d-018f-45bd-853b-78347634f978 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:59.656047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314ba8d-018f-45bd-853b-78347634f978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:59.702836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2314ba8d-018f-45bd-853b-78347634f978 None None] MFA Rules not processed for user `e42fb05dbae64c758098c591d6ffa790`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:59.704743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2314ba8d-018f-45bd-853b-78347634f978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:59.719640 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d7a3d-78c2-4121-970c-19737038cca1 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:59.722371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-581d7a3d-78c2-4121-970c-19737038cca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:59.821222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d7a3d-78c2-4121-970c-19737038cca1 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:59.826380 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2c5e5-3c36-46b0-b8a1-45f49501167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:59.827126 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2c5e5-3c36-46b0-b8a1-45f49501167c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:59.827509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2c5e5-3c36-46b0-b8a1-45f49501167c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:59.827897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2c5e5-3c36-46b0-b8a1-45f49501167c admin admin] PATH_INFO: `/v3/users/55d8e06905644ac39777328db72526ca` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:59.829338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2c5e5-3c36-46b0-b8a1-45f49501167c admin admin] RBAC: Authorizing `identity:delete_user(user_id=55d8e06905644ac39777328db72526ca)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:59.830624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1084/2186] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 13:53:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:59.838478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2c5e5-3c36-46b0-b8a1-45f49501167c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:59.843773 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314ba8d-018f-45bd-853b-78347634f978 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:59.845675 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2314ba8d-018f-45bd-853b-78347634f978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:59.872237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314ba8d-018f-45bd-853b-78347634f978 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:59.880779 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2ea10-5ab1-4413-b18c-acab80372e81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:59.882396 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2ea10-5ab1-4413-b18c-acab80372e81 service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:59.882849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2ea10-5ab1-4413-b18c-acab80372e81 service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:59.883232 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2ea10-5ab1-4413-b18c-acab80372e81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:59.896271 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2ea10-5ab1-4413-b18c-acab80372e81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:59.902714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2ea10-5ab1-4413-b18c-acab80372e81 service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:59.912551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-36b2c5e5-3c36-46b0-b8a1-45f49501167c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55d8e06905644ac39777328db72526ca'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:59.923442 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-36b2c5e5-3c36-46b0-b8a1-45f49501167c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55d8e06905644ac39777328db72526ca'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:59.924331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-36b2c5e5-3c36-46b0-b8a1-45f49501167c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55d8e06905644ac39777328db72526ca'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:59.924977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-36b2c5e5-3c36-46b0-b8a1-45f49501167c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55d8e06905644ac39777328db72526ca'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:53:59.930602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1085/2187] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:59.947335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-36b2c5e5-3c36-46b0-b8a1-45f49501167c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55d8e06905644ac39777328db72526ca'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:00.007883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1103/2188] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:53:59 2022] DELETE /identity/v3/users/55d8e06905644ac39777328db72526ca => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:00.226459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2ea10-5ab1-4413-b18c-acab80372e81 service placement] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:00.263232 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2ea10-5ab1-4413-b18c-acab80372e81 service placement] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:00.273583 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:00.358005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935fea7d-f1eb-4436-8d79-1f0b68731aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:00.358005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fea7d-f1eb-4436-8d79-1f0b68731aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:00.358225 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fea7d-f1eb-4436-8d79-1f0b68731aef admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:00.358350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fea7d-f1eb-4436-8d79-1f0b68731aef admin admin] PATH_INFO: `/v3/projects/b868b7f8289047369b2fcbb500f353e5` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:00.359762 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fea7d-f1eb-4436-8d79-1f0b68731aef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b868b7f8289047369b2fcbb500f353e5)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:00.362191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935fea7d-f1eb-4436-8d79-1f0b68731aef admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:00.490375 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-935fea7d-f1eb-4436-8d79-1f0b68731aef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b868b7f8289047369b2fcbb500f353e5', 'request_id': 'req-935fea7d-f1eb-4436-8d79-1f0b68731aef'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:00.507447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-935fea7d-f1eb-4436-8d79-1f0b68731aef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b868b7f8289047369b2fcbb500f353e5', 'request_id': 'req-935fea7d-f1eb-4436-8d79-1f0b68731aef'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:00.524175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-935fea7d-f1eb-4436-8d79-1f0b68731aef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b868b7f8289047369b2fcbb500f353e5', 'request_id': 'req-935fea7d-f1eb-4436-8d79-1f0b68731aef'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:00.569213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-935fea7d-f1eb-4436-8d79-1f0b68731aef admin admin] The token cache is being invalidate because project b868b7f8289047369b2fcbb500f353e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:00.569784 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-935fea7d-f1eb-4436-8d79-1f0b68731aef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:00.574074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-935fea7d-f1eb-4436-8d79-1f0b68731aef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:00.581316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1086/2189] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:00 2022] DELETE /identity/v3/projects/b868b7f8289047369b2fcbb500f353e5 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:00.584152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23039299-f543-48de-99be-21c408b12962 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:00.584890 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23039299-f543-48de-99be-21c408b12962 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:00.585401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23039299-f543-48de-99be-21c408b12962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:00.627609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-23039299-f543-48de-99be-21c408b12962 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:00.628614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-23039299-f543-48de-99be-21c408b12962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:00.742609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:00.778754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1104/2190] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:00.780378 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e07f2-d84f-431c-83f9-68fd2756e5f5 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:00.781153 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e07f2-d84f-431c-83f9-68fd2756e5f5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:00.781153 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e07f2-d84f-431c-83f9-68fd2756e5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:00.806897 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-843e07f2-d84f-431c-83f9-68fd2756e5f5 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:00.808172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-843e07f2-d84f-431c-83f9-68fd2756e5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:00.821141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1087/2191] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:00.822933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23039299-f543-48de-99be-21c408b12962 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:00.839672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23039299-f543-48de-99be-21c408b12962 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:00.850337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd690-1196-465a-af4a-74e9d3edea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:00.851014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd690-1196-465a-af4a-74e9d3edea59 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:00.851301 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd690-1196-465a-af4a-74e9d3edea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:00.851576 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd690-1196-465a-af4a-74e9d3edea59 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:00.852327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffd690-1196-465a-af4a-74e9d3edea59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:00.853335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffd690-1196-465a-af4a-74e9d3edea59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:00.856431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffd690-1196-465a-af4a-74e9d3edea59 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:00.874730 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1105/2192] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:00.879007 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e07f2-d84f-431c-83f9-68fd2756e5f5 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:00.889772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e07f2-d84f-431c-83f9-68fd2756e5f5 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:00.895184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb29ea-8b40-4bc1-a6a6-5ff497aa5db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:00.895849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb29ea-8b40-4bc1-a6a6-5ff497aa5db2 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:00.896132 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb29ea-8b40-4bc1-a6a6-5ff497aa5db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:00.896591 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb29ea-8b40-4bc1-a6a6-5ff497aa5db2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:00.897220 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb29ea-8b40-4bc1-a6a6-5ff497aa5db2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:00.899157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb29ea-8b40-4bc1-a6a6-5ff497aa5db2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:00.981323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1088/2193] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:00 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:00.985570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd690-1196-465a-af4a-74e9d3edea59 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:00.999879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd690-1196-465a-af4a-74e9d3edea59 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:01.077391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ae724-1d08-44db-ad59-2fa47fa1ee9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:01.078472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ae724-1d08-44db-ad59-2fa47fa1ee9c admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:01.078849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ae724-1d08-44db-ad59-2fa47fa1ee9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:01.079189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ae724-1d08-44db-ad59-2fa47fa1ee9c admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:01.080044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ae724-1d08-44db-ad59-2fa47fa1ee9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:01.082084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916ae724-1d08-44db-ad59-2fa47fa1ee9c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:01.129760 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1106/2194] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:00 2022] POST /identity/v3/users => generated 295 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:01.131308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb29ea-8b40-4bc1-a6a6-5ff497aa5db2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:01.152412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb29ea-8b40-4bc1-a6a6-5ff497aa5db2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:01.157070 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfdc29-1792-435e-ae5c-4c60f02bfbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:01.157858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfdc29-1792-435e-ae5c-4c60f02bfbc3 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:01.158236 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfdc29-1792-435e-ae5c-4c60f02bfbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:01.158634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfdc29-1792-435e-ae5c-4c60f02bfbc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:01.159685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfdc29-1792-435e-ae5c-4c60f02bfbc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:01.162337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfdc29-1792-435e-ae5c-4c60f02bfbc3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:01.175015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1089/2195] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:01.178844 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ae724-1d08-44db-ad59-2fa47fa1ee9c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:01.190947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ae724-1d08-44db-ad59-2fa47fa1ee9c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:01.196123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399b0ac-e211-40c2-a0ae-c953124f342f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:01.196852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399b0ac-e211-40c2-a0ae-c953124f342f admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:01.197225 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399b0ac-e211-40c2-a0ae-c953124f342f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:01.197594 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399b0ac-e211-40c2-a0ae-c953124f342f admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:01.198382 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399b0ac-e211-40c2-a0ae-c953124f342f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:01.200091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a399b0ac-e211-40c2-a0ae-c953124f342f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:01.212025 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1107/2196] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:01.217395 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfdc29-1792-435e-ae5c-4c60f02bfbc3 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:01.233471 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfdc29-1792-435e-ae5c-4c60f02bfbc3 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:01.240268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452e513-ca2c-4d20-b5cf-2db780308928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:01.241075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452e513-ca2c-4d20-b5cf-2db780308928 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:01.241361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452e513-ca2c-4d20-b5cf-2db780308928 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:01.241640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452e513-ca2c-4d20-b5cf-2db780308928 admin admin] PATH_INFO: `/v3/projects/bc4476b1f4b84b5593ff2650592b9fb7/users/75642df28f5d4301b979f39482a9a11d/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:01.264359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452e513-ca2c-4d20-b5cf-2db780308928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4476b1f4b84b5593ff2650592b9fb7, user_id=75642df28f5d4301b979f39482a9a11d, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:01.267649 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452e513-ca2c-4d20-b5cf-2db780308928 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:01.288844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1090/2197] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:01 2022] PUT /identity/v3/projects/bc4476b1f4b84b5593ff2650592b9fb7/users/75642df28f5d4301b979f39482a9a11d/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:01.294088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5c520-bf03-49ec-b934-d04833bdd2b8 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:01.294538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5c520-bf03-49ec-b934-d04833bdd2b8 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:01.294879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5c520-bf03-49ec-b934-d04833bdd2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:01.330119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-43d5c520-bf03-49ec-b934-d04833bdd2b8 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:01.331767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-43d5c520-bf03-49ec-b934-d04833bdd2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:01.337439 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9075fe7-5674-473a-8e26-fa0feec9fc59 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:01.337721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9075fe7-5674-473a-8e26-fa0feec9fc59 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:01.338019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9075fe7-5674-473a-8e26-fa0feec9fc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:01.394304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b9075fe7-5674-473a-8e26-fa0feec9fc59 None None] MFA Rules not processed for user `bed19e9a74be46b5bc836c3cd605133f`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:01.396696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b9075fe7-5674-473a-8e26-fa0feec9fc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:01.506365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1108/2198] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:01.511594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:01.521895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5c520-bf03-49ec-b934-d04833bdd2b8 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:01.523447 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-43d5c520-bf03-49ec-b934-d04833bdd2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:01.579927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1091/2199] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:01.626463 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5c520-bf03-49ec-b934-d04833bdd2b8 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:01.630648 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e05c6-5539-4fca-bc8f-b9bf47e3f601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:01.631235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e05c6-5539-4fca-bc8f-b9bf47e3f601 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:01.631521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e05c6-5539-4fca-bc8f-b9bf47e3f601 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:01.632179 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e05c6-5539-4fca-bc8f-b9bf47e3f601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:01.644357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e05c6-5539-4fca-bc8f-b9bf47e3f601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:01.646165 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e05c6-5539-4fca-bc8f-b9bf47e3f601 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:01.664855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1109/2200] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:54:02.824888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9075fe7-5674-473a-8e26-fa0feec9fc59 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:02.827071 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b9075fe7-5674-473a-8e26-fa0feec9fc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:02.853590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9075fe7-5674-473a-8e26-fa0feec9fc59 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:02.867355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e93dd3d-96f4-47d7-b4d8-cb872d1e7d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:02.869547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93dd3d-96f4-47d7-b4d8-cb872d1e7d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:02.869872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93dd3d-96f4-47d7-b4d8-cb872d1e7d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:02.870147 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e93dd3d-96f4-47d7-b4d8-cb872d1e7d0a admin admin] PATH_INFO: `/v3/users/208a61853ba047028e682f4adfd65eca` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:02.875968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93dd3d-96f4-47d7-b4d8-cb872d1e7d0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=208a61853ba047028e682f4adfd65eca)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:02.878458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e93dd3d-96f4-47d7-b4d8-cb872d1e7d0a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:02.979647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7e93dd3d-96f4-47d7-b4d8-cb872d1e7d0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '208a61853ba047028e682f4adfd65eca'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:02.992349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7e93dd3d-96f4-47d7-b4d8-cb872d1e7d0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '208a61853ba047028e682f4adfd65eca'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:02.993080 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7e93dd3d-96f4-47d7-b4d8-cb872d1e7d0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '208a61853ba047028e682f4adfd65eca'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:02.994346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7e93dd3d-96f4-47d7-b4d8-cb872d1e7d0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '208a61853ba047028e682f4adfd65eca'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:03.004746 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7e93dd3d-96f4-47d7-b4d8-cb872d1e7d0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '208a61853ba047028e682f4adfd65eca'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:03.020857 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e05c6-5539-4fca-bc8f-b9bf47e3f601 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:03.022848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-780e05c6-5539-4fca-bc8f-b9bf47e3f601 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:03.033775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1092/2201] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:02 2022] DELETE /identity/v3/users/208a61853ba047028e682f4adfd65eca => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:03.037092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fffd1-9707-43ed-b254-c528a43b95a0 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:03.037386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fffd1-9707-43ed-b254-c528a43b95a0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:03.037666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fffd1-9707-43ed-b254-c528a43b95a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:03.038246 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:03.049454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e05c6-5539-4fca-bc8f-b9bf47e3f601 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:03.090817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-972fffd1-9707-43ed-b254-c528a43b95a0 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:03.095378 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-972fffd1-9707-43ed-b254-c528a43b95a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:03.150245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3834a26-5792-43de-9461-833b5df03b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:03.151180 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3834a26-5792-43de-9461-833b5df03b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:03.151493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3834a26-5792-43de-9461-833b5df03b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:03.151903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3834a26-5792-43de-9461-833b5df03b61 admin admin] PATH_INFO: `/v3/users/8b445ff1440541198b5d48aaab3b6a63` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:03.153971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3834a26-5792-43de-9461-833b5df03b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b445ff1440541198b5d48aaab3b6a63)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:03.156404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3834a26-5792-43de-9461-833b5df03b61 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:03.177709 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1093/2202] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:03.191567 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fffd1-9707-43ed-b254-c528a43b95a0 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:03.194722 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-972fffd1-9707-43ed-b254-c528a43b95a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:03.270675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c3834a26-5792-43de-9461-833b5df03b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b445ff1440541198b5d48aaab3b6a63'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:03.282318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c3834a26-5792-43de-9461-833b5df03b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b445ff1440541198b5d48aaab3b6a63'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:03.283916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c3834a26-5792-43de-9461-833b5df03b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b445ff1440541198b5d48aaab3b6a63'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:03.284775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c3834a26-5792-43de-9461-833b5df03b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b445ff1440541198b5d48aaab3b6a63'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:03.294194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fffd1-9707-43ed-b254-c528a43b95a0 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:03.297381 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c3834a26-5792-43de-9461-833b5df03b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b445ff1440541198b5d48aaab3b6a63'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:03.300259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2cbc4-3847-4304-8ebf-1d2c0c36cdde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:03.301767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2cbc4-3847-4304-8ebf-1d2c0c36cdde service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:03.303201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2cbc4-3847-4304-8ebf-1d2c0c36cdde service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:03.303555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2cbc4-3847-4304-8ebf-1d2c0c36cdde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:03.305681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-75a2cbc4-3847-4304-8ebf-1d2c0c36cdde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:03.318466 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:03.321233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2cbc4-3847-4304-8ebf-1d2c0c36cdde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:03.323540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2cbc4-3847-4304-8ebf-1d2c0c36cdde service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:03.326787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1110/2203] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:03 2022] DELETE /identity/v3/users/8b445ff1440541198b5d48aaab3b6a63 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:03.331342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a6837-2497-44a6-a8be-ff13f03b06ad None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:03.331831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a6837-2497-44a6-a8be-ff13f03b06ad None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:03.332250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a6837-2497-44a6-a8be-ff13f03b06ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:03.338788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1094/2204] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:03.356638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-264a6837-2497-44a6-a8be-ff13f03b06ad None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:03.358879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-264a6837-2497-44a6-a8be-ff13f03b06ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:03.546687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1111/2205] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:03.604363 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2cbc4-3847-4304-8ebf-1d2c0c36cdde service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:03.606052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-75a2cbc4-3847-4304-8ebf-1d2c0c36cdde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:03.627464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2cbc4-3847-4304-8ebf-1d2c0c36cdde service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:03.745445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21ab8b-7bc4-44ec-9f9c-ee47d25d4ac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:03.746231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21ab8b-7bc4-44ec-9f9c-ee47d25d4ac8 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:03.746507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21ab8b-7bc4-44ec-9f9c-ee47d25d4ac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:03.746778 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21ab8b-7bc4-44ec-9f9c-ee47d25d4ac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:03.771216 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21ab8b-7bc4-44ec-9f9c-ee47d25d4ac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:03.772647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21ab8b-7bc4-44ec-9f9c-ee47d25d4ac8 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:03.832566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1095/2206] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:03.842233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db566f-89ed-404e-ba65-584e945ffb07 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:03.842726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db566f-89ed-404e-ba65-584e945ffb07 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:03.843700 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db566f-89ed-404e-ba65-584e945ffb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:03.882578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-a9db566f-89ed-404e-ba65-584e945ffb07 None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:03.886331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a9db566f-89ed-404e-ba65-584e945ffb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:03.932140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:03.969152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21ab8b-7bc4-44ec-9f9c-ee47d25d4ac8 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:03.970950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8b21ab8b-7bc4-44ec-9f9c-ee47d25d4ac8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:03.991612 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21ab8b-7bc4-44ec-9f9c-ee47d25d4ac8 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:04.025888 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06594f-4727-4ff1-96d1-9a870c22b30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:04.026665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06594f-4727-4ff1-96d1-9a870c22b30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.027085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06594f-4727-4ff1-96d1-9a870c22b30e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.027362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06594f-4727-4ff1-96d1-9a870c22b30e admin admin] PATH_INFO: `/v3/projects/c0846ec480d240c98847746726f43813` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.027773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1112/2207] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 13:54:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:04.028846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06594f-4727-4ff1-96d1-9a870c22b30e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0846ec480d240c98847746726f43813)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:04.031408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06594f-4727-4ff1-96d1-9a870c22b30e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:04.040677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db566f-89ed-404e-ba65-584e945ffb07 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:04.042491 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a9db566f-89ed-404e-ba65-584e945ffb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:04.097312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2e06594f-4727-4ff1-96d1-9a870c22b30e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0846ec480d240c98847746726f43813', 'request_id': 'req-2e06594f-4727-4ff1-96d1-9a870c22b30e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:04.104274 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2e06594f-4727-4ff1-96d1-9a870c22b30e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0846ec480d240c98847746726f43813', 'request_id': 'req-2e06594f-4727-4ff1-96d1-9a870c22b30e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:04.111145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2e06594f-4727-4ff1-96d1-9a870c22b30e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0846ec480d240c98847746726f43813', 'request_id': 'req-2e06594f-4727-4ff1-96d1-9a870c22b30e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:04.126083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2e06594f-4727-4ff1-96d1-9a870c22b30e admin admin] The token cache is being invalidate because project c0846ec480d240c98847746726f43813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:04.126379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2e06594f-4727-4ff1-96d1-9a870c22b30e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:04.127025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2e06594f-4727-4ff1-96d1-9a870c22b30e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:04.129410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1096/2208] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:03 2022] DELETE /identity/v3/projects/c0846ec480d240c98847746726f43813 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:04.133186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db566f-89ed-404e-ba65-584e945ffb07 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:04.134316 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7beef92-d3c4-490c-ae63-a70e9655aa28 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.134603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7beef92-d3c4-490c-ae63-a70e9655aa28 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.134879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7beef92-d3c4-490c-ae63-a70e9655aa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.160948 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f7beef92-d3c4-490c-ae63-a70e9655aa28 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:04.160948 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f7beef92-d3c4-490c-ae63-a70e9655aa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:04.231219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0179663-9d11-45d4-8ce8-dd50cd35f37b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:04.231832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0179663-9d11-45d4-8ce8-dd50cd35f37b service swift] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.232113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0179663-9d11-45d4-8ce8-dd50cd35f37b service swift] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.232412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0179663-9d11-45d4-8ce8-dd50cd35f37b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.234587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c0179663-9d11-45d4-8ce8-dd50cd35f37b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:04.249366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1097/2209] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:04.259631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa7ddd-b0fc-4a05-b87a-1e740c6b079e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.260594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa7ddd-b0fc-4a05-b87a-1e740c6b079e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.260950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa7ddd-b0fc-4a05-b87a-1e740c6b079e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.294947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-42fa7ddd-b0fc-4a05-b87a-1e740c6b079e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:04.296525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-42fa7ddd-b0fc-4a05-b87a-1e740c6b079e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:04.307479 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1098/2210] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:04.311498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa7ddd-b0fc-4a05-b87a-1e740c6b079e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:04.322485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa7ddd-b0fc-4a05-b87a-1e740c6b079e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:04.324876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0179663-9d11-45d4-8ce8-dd50cd35f37b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:04.328769 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493ce1c-a23c-444b-b8d6-152e207dd490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:04.329541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493ce1c-a23c-444b-b8d6-152e207dd490 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.329920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493ce1c-a23c-444b-b8d6-152e207dd490 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.330268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493ce1c-a23c-444b-b8d6-152e207dd490 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.331045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493ce1c-a23c-444b-b8d6-152e207dd490 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:04.331321 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493ce1c-a23c-444b-b8d6-152e207dd490 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:04.333313 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493ce1c-a23c-444b-b8d6-152e207dd490 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:04.334884 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0179663-9d11-45d4-8ce8-dd50cd35f37b service swift] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:04.346197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1113/2211] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 13:54:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 13:54:04.351722 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1099/2212] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:04.352406 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0179663-9d11-45d4-8ce8-dd50cd35f37b service swift] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:04.373071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0179663-9d11-45d4-8ce8-dd50cd35f37b service swift] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:04.378523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1676ad-16f6-4abe-91c6-d616c22ae3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:04.379273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1676ad-16f6-4abe-91c6-d616c22ae3fd admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.379558 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1676ad-16f6-4abe-91c6-d616c22ae3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.379828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1676ad-16f6-4abe-91c6-d616c22ae3fd admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.381084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1676ad-16f6-4abe-91c6-d616c22ae3fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:04.383028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1676ad-16f6-4abe-91c6-d616c22ae3fd admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:04.402441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:04.429815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1114/2213] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:04 2022] POST /identity/v3/projects => generated 381 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:04.434657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493ce1c-a23c-444b-b8d6-152e207dd490 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:04.442162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:04.454987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493ce1c-a23c-444b-b8d6-152e207dd490 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:04.555408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc4ab2-1b86-427b-85bb-bf4102730136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:04.556666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc4ab2-1b86-427b-85bb-bf4102730136 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.557165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc4ab2-1b86-427b-85bb-bf4102730136 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.557618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc4ab2-1b86-427b-85bb-bf4102730136 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.559019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc4ab2-1b86-427b-85bb-bf4102730136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:04.562842 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc4ab2-1b86-427b-85bb-bf4102730136 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:04.598634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1100/2214] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:04 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:04.599162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1676ad-16f6-4abe-91c6-d616c22ae3fd admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:04.615821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1676ad-16f6-4abe-91c6-d616c22ae3fd admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:04.626972 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eaa22d-65bd-4434-803f-7fa78f03e749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:04.627768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eaa22d-65bd-4434-803f-7fa78f03e749 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.628098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eaa22d-65bd-4434-803f-7fa78f03e749 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.628372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eaa22d-65bd-4434-803f-7fa78f03e749 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.629036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eaa22d-65bd-4434-803f-7fa78f03e749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:04.630854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eaa22d-65bd-4434-803f-7fa78f03e749 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:04.643778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1115/2215] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:04.647507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc4ab2-1b86-427b-85bb-bf4102730136 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:04.659077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc4ab2-1b86-427b-85bb-bf4102730136 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:04.664298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22deff9c-a6a8-4649-bde9-c9306766b0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:04.664932 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22deff9c-a6a8-4649-bde9-c9306766b0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.665214 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22deff9c-a6a8-4649-bde9-c9306766b0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.665485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22deff9c-a6a8-4649-bde9-c9306766b0c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.666180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22deff9c-a6a8-4649-bde9-c9306766b0c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:04.667795 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22deff9c-a6a8-4649-bde9-c9306766b0c8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:04.687130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1101/2216] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:04.687510 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eaa22d-65bd-4434-803f-7fa78f03e749 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:04.703412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eaa22d-65bd-4434-803f-7fa78f03e749 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:04.728845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c47ba-ba88-4b2b-82c5-bec95eee1578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:04.729530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c47ba-ba88-4b2b-82c5-bec95eee1578 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.729877 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c47ba-ba88-4b2b-82c5-bec95eee1578 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.730329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c47ba-ba88-4b2b-82c5-bec95eee1578 admin admin] PATH_INFO: `/v3/projects/0375e7ef3fb04600b5d4095057658bc7/users/e93b5304ecec4ceca33da4b58e8e2480/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.779415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c47ba-ba88-4b2b-82c5-bec95eee1578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0375e7ef3fb04600b5d4095057658bc7, user_id=e93b5304ecec4ceca33da4b58e8e2480, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:04.782154 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c47ba-ba88-4b2b-82c5-bec95eee1578 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:04.811893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1116/2217] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:04 2022] PUT /identity/v3/projects/0375e7ef3fb04600b5d4095057658bc7/users/e93b5304ecec4ceca33da4b58e8e2480/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:04.817650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4b2e-0045-47e7-993b-87fe3977b39e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:04.818244 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4b2e-0045-47e7-993b-87fe3977b39e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:04.818859 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4b2e-0045-47e7-993b-87fe3977b39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:04.864406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-309d4b2e-0045-47e7-993b-87fe3977b39e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:04.870349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-309d4b2e-0045-47e7-993b-87fe3977b39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:04.960647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c47ba-ba88-4b2b-82c5-bec95eee1578 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:04.964643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f48c47ba-ba88-4b2b-82c5-bec95eee1578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:05.063301 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1102/2218] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:05.074246 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d4b2e-0045-47e7-993b-87fe3977b39e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:05.077021 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-309d4b2e-0045-47e7-993b-87fe3977b39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:05.134848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c47ba-ba88-4b2b-82c5-bec95eee1578 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:05.142174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:05.143183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:05.143413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:05.143978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] PATH_INFO: `/v3/projects/de1abbe94e2348c4831f2efd35fec25e` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:05.145362 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1abbe94e2348c4831f2efd35fec25e)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:05.147705 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:05.172949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d4b2e-0045-47e7-993b-87fe3977b39e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:05.178721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11be03-db29-4a13-9de8-c0e0b7604e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:05.179484 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11be03-db29-4a13-9de8-c0e0b7604e56 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:05.179846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11be03-db29-4a13-9de8-c0e0b7604e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:05.180277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11be03-db29-4a13-9de8-c0e0b7604e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:05.203229 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11be03-db29-4a13-9de8-c0e0b7604e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:05.204808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11be03-db29-4a13-9de8-c0e0b7604e56 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:05.234753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1103/2219] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:05.237476 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefd6b3-1e9a-4503-961e-feebc21834c9 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:05.237789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefd6b3-1e9a-4503-961e-feebc21834c9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:05.238688 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefd6b3-1e9a-4503-961e-feebc21834c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:05.284320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7fefd6b3-1e9a-4503-961e-feebc21834c9 None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:05.285896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7fefd6b3-1e9a-4503-961e-feebc21834c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:05.297061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1abbe94e2348c4831f2efd35fec25e', 'request_id': 'req-4ed9f1de-c968-441c-bb89-f687cefa6845'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:05.313325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1abbe94e2348c4831f2efd35fec25e', 'request_id': 'req-4ed9f1de-c968-441c-bb89-f687cefa6845'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:05.328275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1abbe94e2348c4831f2efd35fec25e', 'request_id': 'req-4ed9f1de-c968-441c-bb89-f687cefa6845'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:05.333855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:05.355344 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] The token cache is being invalidate because project de1abbe94e2348c4831f2efd35fec25e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:05.355697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:05.360631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:05.362950 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1117/2220] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:04 2022] DELETE /identity/v3/projects/de1abbe94e2348c4831f2efd35fec25e => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:05.367083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:05.370805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:05.421064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1104/2221] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 13:54:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:05.441043 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefd6b3-1e9a-4503-961e-feebc21834c9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:05.442702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7fefd6b3-1e9a-4503-961e-feebc21834c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:05.447367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:05.507613 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9f1de-c968-441c-bb89-f687cefa6845 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:05.535353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:05.536263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:05.536544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:05.536815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] PATH_INFO: `/v3/users/21451dfe16554b24ac91aaeb9439d793` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:05.539579 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21451dfe16554b24ac91aaeb9439d793)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:05.542114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:05.619336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefd6b3-1e9a-4503-961e-feebc21834c9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:05.632604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21451dfe16554b24ac91aaeb9439d793'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:05.633826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ba268-bf39-4cd4-a686-ed57c812c04b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:05.636230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ba268-bf39-4cd4-a686-ed57c812c04b service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:05.637425 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ba268-bf39-4cd4-a686-ed57c812c04b service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:05.638251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ba268-bf39-4cd4-a686-ed57c812c04b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:05.648657 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21451dfe16554b24ac91aaeb9439d793'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:05.649936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21451dfe16554b24ac91aaeb9439d793'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:05.651955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21451dfe16554b24ac91aaeb9439d793'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:05.659731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ba268-bf39-4cd4-a686-ed57c812c04b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:05.662657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ba268-bf39-4cd4-a686-ed57c812c04b service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:05.668806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21451dfe16554b24ac91aaeb9439d793'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:05.688794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1105/2222] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 13:54:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 13:54:05.693956 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563b025-73ac-4e01-bcd8-366a4629da0e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:05.700213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563b025-73ac-4e01-bcd8-366a4629da0e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:05.700554 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c563b025-73ac-4e01-bcd8-366a4629da0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:05.721012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1118/2223] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:05 2022] DELETE /identity/v3/users/21451dfe16554b24ac91aaeb9439d793 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:05.739281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c563b025-73ac-4e01-bcd8-366a4629da0e None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:05.740849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c563b025-73ac-4e01-bcd8-366a4629da0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:05.860997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1106/2224] 10.222.0.55 () {58 vars in 967 bytes} [Thu Sep 8 13:54:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:05.932058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:05.936624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:06.120114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0353e9f-28e6-4459-b9ae-42e7643ef647 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:06.131767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161f736-22d9-43f6-8dab-bddd9368e1b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:06.132933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161f736-22d9-43f6-8dab-bddd9368e1b0 service swift] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:06.133314 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161f736-22d9-43f6-8dab-bddd9368e1b0 service swift] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:06.142611 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161f736-22d9-43f6-8dab-bddd9368e1b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:06.168238 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161f736-22d9-43f6-8dab-bddd9368e1b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:06.170431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161f736-22d9-43f6-8dab-bddd9368e1b0 service swift] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:06.189218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1119/2225] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 13:54:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:54:06.802587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563b025-73ac-4e01-bcd8-366a4629da0e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:06.804201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c563b025-73ac-4e01-bcd8-366a4629da0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:06.852490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eddc15-0a10-4af1-b39d-777fb90717e1 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:06.852855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eddc15-0a10-4af1-b39d-777fb90717e1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:06.853199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eddc15-0a10-4af1-b39d-777fb90717e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:06.956946 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-93eddc15-0a10-4af1-b39d-777fb90717e1 None None] MFA Rules not processed for user `e93b5304ecec4ceca33da4b58e8e2480`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:06.959718 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-93eddc15-0a10-4af1-b39d-777fb90717e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:06.994778 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c563b025-73ac-4e01-bcd8-366a4629da0e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:07.004436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:07.005356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:07.005700 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:07.006126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] PATH_INFO: `/v3/projects/e630732383224c92aa1850f04276546b` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:07.007373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e630732383224c92aa1850f04276546b)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:07.009756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:07.018088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:07.112599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:07.142137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e630732383224c92aa1850f04276546b', 'request_id': 'req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:07.155623 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e630732383224c92aa1850f04276546b', 'request_id': 'req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:07.158143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1120/2226] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:07.160162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20e24d-e16d-42d2-a848-cea8706a8503 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:07.160542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20e24d-e16d-42d2-a848-cea8706a8503 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:07.160825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee20e24d-e16d-42d2-a848-cea8706a8503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:07.177868 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e630732383224c92aa1850f04276546b', 'request_id': 'req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:07.200271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-ee20e24d-e16d-42d2-a848-cea8706a8503 None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:07.203138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ee20e24d-e16d-42d2-a848-cea8706a8503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:07.214798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] The token cache is being invalidate because project e630732383224c92aa1850f04276546b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:07.215106 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:07.217583 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:07.219596 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1107/2227] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:06 2022] DELETE /identity/v3/projects/e630732383224c92aa1850f04276546b => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:07.223056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:07.229473 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:07.332510 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1121/2228] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 13:54:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:07.334184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6564a5-d6f6-4baf-9445-d1a19f03aca7 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:07.334568 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6564a5-d6f6-4baf-9445-d1a19f03aca7 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:07.334936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6564a5-d6f6-4baf-9445-d1a19f03aca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:07.371351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-ff6564a5-d6f6-4baf-9445-d1a19f03aca7 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:07.375292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ff6564a5-d6f6-4baf-9445-d1a19f03aca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:07.476111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1afa3b-c86d-40b2-aeb8-1a2b14b3e143 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:07.487705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611bb16-ebe4-438d-8ee9-aa4e4ae6eec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:07.488607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611bb16-ebe4-438d-8ee9-aa4e4ae6eec8 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:07.489064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611bb16-ebe4-438d-8ee9-aa4e4ae6eec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:07.489485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611bb16-ebe4-438d-8ee9-aa4e4ae6eec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:07.508467 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3611bb16-ebe4-438d-8ee9-aa4e4ae6eec8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:07.542354 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1122/2229] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:07.542354 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6564a5-d6f6-4baf-9445-d1a19f03aca7 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:07.543141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ff6564a5-d6f6-4baf-9445-d1a19f03aca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:07.688773 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611bb16-ebe4-438d-8ee9-aa4e4ae6eec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:07.690510 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611bb16-ebe4-438d-8ee9-aa4e4ae6eec8 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:07.710910 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6564a5-d6f6-4baf-9445-d1a19f03aca7 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:07.717661 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1108/2230] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:07.720548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44b2ff-4fd5-45bb-822f-e4a607210d50 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:07.721885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44b2ff-4fd5-45bb-822f-e4a607210d50 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:07.722793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44b2ff-4fd5-45bb-822f-e4a607210d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:07.728286 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad45af-9b26-4874-82f0-d0cb85c08228 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:07.729449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad45af-9b26-4874-82f0-d0cb85c08228 service swift] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:07.729897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad45af-9b26-4874-82f0-d0cb85c08228 service swift] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:07.730308 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad45af-9b26-4874-82f0-d0cb85c08228 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:07.750358 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad45af-9b26-4874-82f0-d0cb85c08228 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:07.753476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad45af-9b26-4874-82f0-d0cb85c08228 service swift] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:07.776982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-3a44b2ff-4fd5-45bb-822f-e4a607210d50 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:07.778237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3a44b2ff-4fd5-45bb-822f-e4a607210d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:07.792939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1123/2231] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 13:54:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:54:07.805095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1109/2232] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:07.810058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad45af-9b26-4874-82f0-d0cb85c08228 service swift] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:07.821907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad45af-9b26-4874-82f0-d0cb85c08228 service swift] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:07.828624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9388d0-3d1f-4b34-9242-71196fddb66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:07.829241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9388d0-3d1f-4b34-9242-71196fddb66f admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:07.829519 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9388d0-3d1f-4b34-9242-71196fddb66f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:07.831091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9388d0-3d1f-4b34-9242-71196fddb66f admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:07.831091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9388d0-3d1f-4b34-9242-71196fddb66f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:07.831444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9388d0-3d1f-4b34-9242-71196fddb66f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:07.833542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9388d0-3d1f-4b34-9242-71196fddb66f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:07.861408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44b2ff-4fd5-45bb-822f-e4a607210d50 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:07.867136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1124/2233] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:07.872284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44b2ff-4fd5-45bb-822f-e4a607210d50 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:07.915542 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e5683-941f-4b48-ad00-0c48b3d1acf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:07.916187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e5683-941f-4b48-ad00-0c48b3d1acf7 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:07.916463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e5683-941f-4b48-ad00-0c48b3d1acf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:07.916731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e5683-941f-4b48-ad00-0c48b3d1acf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:07.917384 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e5683-941f-4b48-ad00-0c48b3d1acf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:07.919310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e5683-941f-4b48-ad00-0c48b3d1acf7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:08.008573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1110/2234] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:07 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:08.010769 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9388d0-3d1f-4b34-9242-71196fddb66f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:08.025896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9388d0-3d1f-4b34-9242-71196fddb66f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:08.040813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:08.209124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b34676-72fa-461a-a2e1-5a07820bb122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:08.210688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b34676-72fa-461a-a2e1-5a07820bb122 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:08.210688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b34676-72fa-461a-a2e1-5a07820bb122 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:08.210931 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b34676-72fa-461a-a2e1-5a07820bb122 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:08.214961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b34676-72fa-461a-a2e1-5a07820bb122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:08.217701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b34676-72fa-461a-a2e1-5a07820bb122 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:08.286580 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1125/2235] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:08 2022] POST /identity/v3/users => generated 293 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:08.290732 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e5683-941f-4b48-ad00-0c48b3d1acf7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:08.319337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e5683-941f-4b48-ad00-0c48b3d1acf7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:08.332182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1c418-85d5-45c5-a03c-0a6a22c18906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:08.340859 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1c418-85d5-45c5-a03c-0a6a22c18906 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:08.341049 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1c418-85d5-45c5-a03c-0a6a22c18906 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:08.341268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1c418-85d5-45c5-a03c-0a6a22c18906 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:08.341988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1c418-85d5-45c5-a03c-0a6a22c18906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:08.346778 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1c418-85d5-45c5-a03c-0a6a22c18906 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:08.387932 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1111/2236] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:08.389536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b34676-72fa-461a-a2e1-5a07820bb122 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:08.406026 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b34676-72fa-461a-a2e1-5a07820bb122 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:08.426135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2987f-a34c-4a33-a117-f2285024c8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:08.427699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2987f-a34c-4a33-a117-f2285024c8ae admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:08.428167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2987f-a34c-4a33-a117-f2285024c8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:08.428541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2987f-a34c-4a33-a117-f2285024c8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:08.429285 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2987f-a34c-4a33-a117-f2285024c8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:08.431198 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2987f-a34c-4a33-a117-f2285024c8ae admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:08.486814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1c418-85d5-45c5-a03c-0a6a22c18906 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:08.487420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1126/2237] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:08.508176 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1c418-85d5-45c5-a03c-0a6a22c18906 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:08.531305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18f35e-d882-4fc1-aae3-38c70494826a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:08.533431 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18f35e-d882-4fc1-aae3-38c70494826a admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:08.534464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18f35e-d882-4fc1-aae3-38c70494826a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:08.535187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18f35e-d882-4fc1-aae3-38c70494826a admin admin] PATH_INFO: `/v3/projects/8b9a99ac35ce4de697592743e3669bf6/users/18a85e492d6b45549eadf668a729b2a7/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:08.544164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:08.589381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18f35e-d882-4fc1-aae3-38c70494826a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b9a99ac35ce4de697592743e3669bf6, user_id=18a85e492d6b45549eadf668a729b2a7, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:08.597283 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18f35e-d882-4fc1-aae3-38c70494826a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:08.630416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1112/2238] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:08 2022] PUT /identity/v3/projects/8b9a99ac35ce4de697592743e3669bf6/users/18a85e492d6b45549eadf668a729b2a7/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:08.642790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98270ee-1a51-4571-a125-16bb3f153c46 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:08.642978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98270ee-1a51-4571-a125-16bb3f153c46 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:08.643272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98270ee-1a51-4571-a125-16bb3f153c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:08.726591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-e98270ee-1a51-4571-a125-16bb3f153c46 None None] MFA Rules not processed for user `18a85e492d6b45549eadf668a729b2a7`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:08.729460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e98270ee-1a51-4571-a125-16bb3f153c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:08.758154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290274c4-ca6c-48b6-ab24-00a48a9c742d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:08.758445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290274c4-ca6c-48b6-ab24-00a48a9c742d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:08.758793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290274c4-ca6c-48b6-ab24-00a48a9c742d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:08.818342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-290274c4-ca6c-48b6-ab24-00a48a9c742d None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:08.821047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-290274c4-ca6c-48b6-ab24-00a48a9c742d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:08.900734 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1127/2239] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:08.905746 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98270ee-1a51-4571-a125-16bb3f153c46 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:08.922777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1113/2240] 10.222.0.55 () {58 vars in 967 bytes} [Thu Sep 8 13:54:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:08.946530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290274c4-ca6c-48b6-ab24-00a48a9c742d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:09.074823 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98270ee-1a51-4571-a125-16bb3f153c46 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:09.081277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c7d63-7526-4028-a8ee-4872f7f3c720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:09.082608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c7d63-7526-4028-a8ee-4872f7f3c720 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.083129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c7d63-7526-4028-a8ee-4872f7f3c720 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.083548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c7d63-7526-4028-a8ee-4872f7f3c720 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.084426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c7d63-7526-4028-a8ee-4872f7f3c720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:09.086999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c7d63-7526-4028-a8ee-4872f7f3c720 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:09.141264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290274c4-ca6c-48b6-ab24-00a48a9c742d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:09.149558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c6cd8-e9f1-40dd-a74b-9805a2782f13 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:09.150687 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c6cd8-e9f1-40dd-a74b-9805a2782f13 service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.151294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c6cd8-e9f1-40dd-a74b-9805a2782f13 service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.151799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c6cd8-e9f1-40dd-a74b-9805a2782f13 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.158483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:09.163826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c6cd8-e9f1-40dd-a74b-9805a2782f13 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:09.166480 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c6cd8-e9f1-40dd-a74b-9805a2782f13 service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:09.178159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1114/2241] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 13:54:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:54:09.184003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1128/2242] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:08 2022] POST /identity/v3/projects => generated 385 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:09.189178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c6cd8-e9f1-40dd-a74b-9805a2782f13 service swift] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:09.204374 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c6cd8-e9f1-40dd-a74b-9805a2782f13 service swift] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:09.297412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518210f-0347-47c7-9d64-b3d7c48d6e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:09.298013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518210f-0347-47c7-9d64-b3d7c48d6e33 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.298350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518210f-0347-47c7-9d64-b3d7c48d6e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.298719 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518210f-0347-47c7-9d64-b3d7c48d6e33 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.307030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518210f-0347-47c7-9d64-b3d7c48d6e33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:09.309552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518210f-0347-47c7-9d64-b3d7c48d6e33 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:09.346076 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1115/2243] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:09 2022] POST /identity/v3/users => generated 299 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:09.350273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c7d63-7526-4028-a8ee-4872f7f3c720 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:09.374008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c7d63-7526-4028-a8ee-4872f7f3c720 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:09.389258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c0a6d-2d08-4b7e-9fa3-3429d62644fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:09.389914 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c0a6d-2d08-4b7e-9fa3-3429d62644fe admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.390368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c0a6d-2d08-4b7e-9fa3-3429d62644fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.390493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c0a6d-2d08-4b7e-9fa3-3429d62644fe admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.394137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c0a6d-2d08-4b7e-9fa3-3429d62644fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:09.395840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c0a6d-2d08-4b7e-9fa3-3429d62644fe admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:09.418950 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1129/2244] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:09.421445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518210f-0347-47c7-9d64-b3d7c48d6e33 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:09.461237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518210f-0347-47c7-9d64-b3d7c48d6e33 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:09.477568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ddbaf-11a9-4d89-8b8e-63d3dffb462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:09.478986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ddbaf-11a9-4d89-8b8e-63d3dffb462d admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.480116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ddbaf-11a9-4d89-8b8e-63d3dffb462d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.480936 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ddbaf-11a9-4d89-8b8e-63d3dffb462d admin admin] PATH_INFO: `/v3/projects/6a7aa4b73eb94e05b9fff207b3b706c1/users/0069795e5d024fb6b1db8abe3db65040/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.487853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:09.505721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ddbaf-11a9-4d89-8b8e-63d3dffb462d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7aa4b73eb94e05b9fff207b3b706c1, user_id=0069795e5d024fb6b1db8abe3db65040, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:09.509895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ddbaf-11a9-4d89-8b8e-63d3dffb462d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:09.560167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1116/2245] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:09 2022] PUT /identity/v3/projects/6a7aa4b73eb94e05b9fff207b3b706c1/users/0069795e5d024fb6b1db8abe3db65040/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:09.562590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c0a6d-2d08-4b7e-9fa3-3429d62644fe admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:09.580098 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c0a6d-2d08-4b7e-9fa3-3429d62644fe admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:09.788248 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9490640-f4b8-4971-9bab-393adc8a89d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:09.788730 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9490640-f4b8-4971-9bab-393adc8a89d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.789011 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9490640-f4b8-4971-9bab-393adc8a89d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.789287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9490640-f4b8-4971-9bab-393adc8a89d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.791237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9490640-f4b8-4971-9bab-393adc8a89d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:09.793954 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9490640-f4b8-4971-9bab-393adc8a89d7 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:09.816725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1130/2246] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:09.818399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ddbaf-11a9-4d89-8b8e-63d3dffb462d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:09.826909 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054cc58-0d32-4bb2-82c2-ed3b0ba00094 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.827191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054cc58-0d32-4bb2-82c2-ed3b0ba00094 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.827472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054cc58-0d32-4bb2-82c2-ed3b0ba00094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.842934 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ddbaf-11a9-4d89-8b8e-63d3dffb462d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:09.852394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df6a1f-d932-4d2a-8b7b-057ed59f5255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:09.853266 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df6a1f-d932-4d2a-8b7b-057ed59f5255 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.853990 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df6a1f-d932-4d2a-8b7b-057ed59f5255 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.854604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df6a1f-d932-4d2a-8b7b-057ed59f5255 admin admin] PATH_INFO: `/v3/projects/6a7aa4b73eb94e05b9fff207b3b706c1/users/0069795e5d024fb6b1db8abe3db65040/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.857942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df6a1f-d932-4d2a-8b7b-057ed59f5255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a7aa4b73eb94e05b9fff207b3b706c1, user_id=0069795e5d024fb6b1db8abe3db65040, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:09.862214 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df6a1f-d932-4d2a-8b7b-057ed59f5255 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:09.874816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-d054cc58-0d32-4bb2-82c2-ed3b0ba00094 None None] MFA Rules not processed for user `e93b5304ecec4ceca33da4b58e8e2480`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:09.879288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d054cc58-0d32-4bb2-82c2-ed3b0ba00094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:09.885563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1117/2247] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:09 2022] PUT /identity/v3/projects/6a7aa4b73eb94e05b9fff207b3b706c1/users/0069795e5d024fb6b1db8abe3db65040/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:09.914682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8598e-8c63-4611-8905-20aa6dbdd0b5 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.914969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8598e-8c63-4611-8905-20aa6dbdd0b5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.915240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8598e-8c63-4611-8905-20aa6dbdd0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.997412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1131/2248] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:10.045470 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-27c8598e-8c63-4611-8905-20aa6dbdd0b5 None None] MFA Rules not processed for user `0069795e5d024fb6b1db8abe3db65040`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:10.047339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-27c8598e-8c63-4611-8905-20aa6dbdd0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:10.171638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1118/2249] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:10.182473 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054cc58-0d32-4bb2-82c2-ed3b0ba00094 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:10.384157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054cc58-0d32-4bb2-82c2-ed3b0ba00094 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:10.394747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f187a-6faa-4878-b1f6-0d9aef27f913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:10.395306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f187a-6faa-4878-b1f6-0d9aef27f913 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:10.395590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f187a-6faa-4878-b1f6-0d9aef27f913 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:10.395861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f187a-6faa-4878-b1f6-0d9aef27f913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:10.452176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f187a-6faa-4878-b1f6-0d9aef27f913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:10.453643 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f187a-6faa-4878-b1f6-0d9aef27f913 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:10.490071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1132/2250] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:11.720855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8598e-8c63-4611-8905-20aa6dbdd0b5 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:11.837614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8598e-8c63-4611-8905-20aa6dbdd0b5 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:11.848949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:11.849833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:11.850227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:11.850603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] PATH_INFO: `/v3/users/18a85e492d6b45549eadf668a729b2a7` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:11.852130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18a85e492d6b45549eadf668a729b2a7)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:11.854520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:11.948674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18a85e492d6b45549eadf668a729b2a7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:11.960905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18a85e492d6b45549eadf668a729b2a7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:11.962205 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18a85e492d6b45549eadf668a729b2a7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:11.963074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18a85e492d6b45549eadf668a729b2a7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:11.969259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:11.987410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18a85e492d6b45549eadf668a729b2a7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:12.038396 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1119/2251] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:11 2022] DELETE /identity/v3/users/18a85e492d6b45549eadf668a729b2a7 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:12.040177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee6718-d536-42f2-8603-485b667f98fc None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:12.040787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee6718-d536-42f2-8603-485b667f98fc None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:12.040787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee6718-d536-42f2-8603-485b667f98fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.087232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-35ee6718-d536-42f2-8603-485b667f98fc None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:12.088457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-35ee6718-d536-42f2-8603-485b667f98fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:12.164263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:12.212749 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1133/2252] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:12.222788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:12.224361 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:12.359903 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91bc7c2-da49-4c7a-9367-a53878ef1902 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:12.369540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b2574-6a93-495a-8e4f-ba3ee20197e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:12.370225 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b2574-6a93-495a-8e4f-ba3ee20197e6 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:12.370477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b2574-6a93-495a-8e4f-ba3ee20197e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:12.370802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b2574-6a93-495a-8e4f-ba3ee20197e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.395625 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b2574-6a93-495a-8e4f-ba3ee20197e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:12.398993 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87b2574-6a93-495a-8e4f-ba3ee20197e6 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:12.424398 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1120/2253] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:54:12.860264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee6718-d536-42f2-8603-485b667f98fc None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:12.862008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-35ee6718-d536-42f2-8603-485b667f98fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:12.883754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee6718-d536-42f2-8603-485b667f98fc None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:12.910742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:12.911365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:12.911895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:12.912259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] PATH_INFO: `/v3/users/e60401b675bb4dff9ebc12b6076fdd3f` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.914421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e60401b675bb4dff9ebc12b6076fdd3f)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:12.917694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:13.043697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e60401b675bb4dff9ebc12b6076fdd3f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.058424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e60401b675bb4dff9ebc12b6076fdd3f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.059038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e60401b675bb4dff9ebc12b6076fdd3f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.059668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e60401b675bb4dff9ebc12b6076fdd3f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.078085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e60401b675bb4dff9ebc12b6076fdd3f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.142129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1134/2254] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:12 2022] DELETE /identity/v3/users/e60401b675bb4dff9ebc12b6076fdd3f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:13.146924 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c8c9a-5ef3-4812-a952-ece60337e8aa None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:13.147458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c8c9a-5ef3-4812-a952-ece60337e8aa None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:13.147458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c8c9a-5ef3-4812-a952-ece60337e8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:13.184065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e16c8c9a-5ef3-4812-a952-ece60337e8aa None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:13.188388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e16c8c9a-5ef3-4812-a952-ece60337e8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:13.334367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1121/2255] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:13.349465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:13.351295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:13.380551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c8c9a-5ef3-4812-a952-ece60337e8aa None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:13.382950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e16c8c9a-5ef3-4812-a952-ece60337e8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:13.411514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c8c9a-5ef3-4812-a952-ece60337e8aa None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:13.425037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c4596-22be-4ecc-acfb-736ef3394ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:13.425725 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c4596-22be-4ecc-acfb-736ef3394ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:13.426035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c4596-22be-4ecc-acfb-736ef3394ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:13.426319 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c4596-22be-4ecc-acfb-736ef3394ea9 admin admin] PATH_INFO: `/v3/projects/8b9a99ac35ce4de697592743e3669bf6` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:13.427644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c4596-22be-4ecc-acfb-736ef3394ea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b9a99ac35ce4de697592743e3669bf6)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:13.429871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c4596-22be-4ecc-acfb-736ef3394ea9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:13.454939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1b0a1-670b-4639-9bda-e911803fd01c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:13.460727 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6855abb-b9de-4963-9f99-08b5975f6b97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:13.461715 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6855abb-b9de-4963-9f99-08b5975f6b97 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:13.462448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6855abb-b9de-4963-9f99-08b5975f6b97 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:13.462904 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6855abb-b9de-4963-9f99-08b5975f6b97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:13.475352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6855abb-b9de-4963-9f99-08b5975f6b97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:13.476932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6855abb-b9de-4963-9f99-08b5975f6b97 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:13.489471 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1135/2256] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:13.541538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8f9c4596-22be-4ecc-acfb-736ef3394ea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b9a99ac35ce4de697592743e3669bf6', 'request_id': 'req-8f9c4596-22be-4ecc-acfb-736ef3394ea9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.558123 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8f9c4596-22be-4ecc-acfb-736ef3394ea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b9a99ac35ce4de697592743e3669bf6', 'request_id': 'req-8f9c4596-22be-4ecc-acfb-736ef3394ea9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.577666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8f9c4596-22be-4ecc-acfb-736ef3394ea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b9a99ac35ce4de697592743e3669bf6', 'request_id': 'req-8f9c4596-22be-4ecc-acfb-736ef3394ea9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.607820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8f9c4596-22be-4ecc-acfb-736ef3394ea9 admin admin] The token cache is being invalidate because project 8b9a99ac35ce4de697592743e3669bf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:13.614533 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8f9c4596-22be-4ecc-acfb-736ef3394ea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.616070 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8f9c4596-22be-4ecc-acfb-736ef3394ea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.620877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1122/2257] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:13 2022] DELETE /identity/v3/projects/8b9a99ac35ce4de697592743e3669bf6 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:13.634932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6855abb-b9de-4963-9f99-08b5975f6b97 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:13.653542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6855abb-b9de-4963-9f99-08b5975f6b97 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:13.711667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:13.728264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:13.728989 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:13.729482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:13.729874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] PATH_INFO: `/v3/users/0069795e5d024fb6b1db8abe3db65040` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:13.733446 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0069795e5d024fb6b1db8abe3db65040)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:13.736110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:13.827458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0069795e5d024fb6b1db8abe3db65040'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.843818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0069795e5d024fb6b1db8abe3db65040'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.844813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0069795e5d024fb6b1db8abe3db65040'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.845528 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0069795e5d024fb6b1db8abe3db65040'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.866690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4065fb3-9b8c-4c5e-9b87-2b20ea036367 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:13.867083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4065fb3-9b8c-4c5e-9b87-2b20ea036367 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:13.867256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4065fb3-9b8c-4c5e-9b87-2b20ea036367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:13.868867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0069795e5d024fb6b1db8abe3db65040'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:13.877939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:13.937152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1136/2258] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:13 2022] DELETE /identity/v3/users/0069795e5d024fb6b1db8abe3db65040 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:13.981134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a4065fb3-9b8c-4c5e-9b87-2b20ea036367 None None] MFA Rules not processed for user `75642df28f5d4301b979f39482a9a11d`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:13.983041 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a4065fb3-9b8c-4c5e-9b87-2b20ea036367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:14.052391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:14.053899 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:14.276071 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1123/2259] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:14.300833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ede020-a5f3-4922-b5c3-7de451a554a0 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:14.314111 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:14.315202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:14.315651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:14.316097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] PATH_INFO: `/v3/projects/06929ea6da40417394a9274b2c86c96a` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:14.317453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06929ea6da40417394a9274b2c86c96a)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:14.320538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:14.358088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4065fb3-9b8c-4c5e-9b87-2b20ea036367 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:14.359857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a4065fb3-9b8c-4c5e-9b87-2b20ea036367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:14.461463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4065fb3-9b8c-4c5e-9b87-2b20ea036367 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:14.462475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06929ea6da40417394a9274b2c86c96a', 'request_id': 'req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:14.479229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06929ea6da40417394a9274b2c86c96a', 'request_id': 'req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:14.496193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06929ea6da40417394a9274b2c86c96a', 'request_id': 'req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:14.508635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:14.522749 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] The token cache is being invalidate because project 06929ea6da40417394a9274b2c86c96a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:14.524015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:14.525976 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:14.546149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1137/2260] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:14 2022] DELETE /identity/v3/projects/06929ea6da40417394a9274b2c86c96a => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:14.589367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e18884-d892-4b33-beae-5c139841b0c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:14.593265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e18884-d892-4b33-beae-5c139841b0c5 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:14.593688 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e18884-d892-4b33-beae-5c139841b0c5 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:14.593997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e18884-d892-4b33-beae-5c139841b0c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:14.599120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a0e18884-d892-4b33-beae-5c139841b0c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:14.672841 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:14.740690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:14.742722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:14.765389 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e18884-d892-4b33-beae-5c139841b0c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:14.767181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e18884-d892-4b33-beae-5c139841b0c5 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:14.793512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1124/2261] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:14.887339 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68d5cc-d627-4e12-b5e0-9d7eddd19407 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:14.894672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954ba88-88b7-4a75-96d8-72c2a7a7df00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:14.895600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954ba88-88b7-4a75-96d8-72c2a7a7df00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:14.896725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954ba88-88b7-4a75-96d8-72c2a7a7df00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:14.896725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954ba88-88b7-4a75-96d8-72c2a7a7df00 admin admin] PATH_INFO: `/v3/projects/6a7aa4b73eb94e05b9fff207b3b706c1` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:14.898001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954ba88-88b7-4a75-96d8-72c2a7a7df00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a7aa4b73eb94e05b9fff207b3b706c1)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:14.900958 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954ba88-88b7-4a75-96d8-72c2a7a7df00 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:14.996826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d954ba88-88b7-4a75-96d8-72c2a7a7df00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a7aa4b73eb94e05b9fff207b3b706c1', 'request_id': 'req-d954ba88-88b7-4a75-96d8-72c2a7a7df00'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:15.028024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d954ba88-88b7-4a75-96d8-72c2a7a7df00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a7aa4b73eb94e05b9fff207b3b706c1', 'request_id': 'req-d954ba88-88b7-4a75-96d8-72c2a7a7df00'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:15.044566 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d954ba88-88b7-4a75-96d8-72c2a7a7df00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a7aa4b73eb94e05b9fff207b3b706c1', 'request_id': 'req-d954ba88-88b7-4a75-96d8-72c2a7a7df00'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:15.079413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d954ba88-88b7-4a75-96d8-72c2a7a7df00 admin admin] The token cache is being invalidate because project 6a7aa4b73eb94e05b9fff207b3b706c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:15.080103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d954ba88-88b7-4a75-96d8-72c2a7a7df00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:15.080493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d954ba88-88b7-4a75-96d8-72c2a7a7df00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:15.086466 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1138/2262] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:14 2022] DELETE /identity/v3/projects/6a7aa4b73eb94e05b9fff207b3b706c1 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:15.087342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12bb11-f51e-4218-b6ee-1da6b4109e71 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.087638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12bb11-f51e-4218-b6ee-1da6b4109e71 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.087912 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12bb11-f51e-4218-b6ee-1da6b4109e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.127163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5b12bb11-f51e-4218-b6ee-1da6b4109e71 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:15.128444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5b12bb11-f51e-4218-b6ee-1da6b4109e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:15.234638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1125/2263] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:15.242796 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca006010-9836-457f-91cd-9ccd503decc0 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.243081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca006010-9836-457f-91cd-9ccd503decc0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.243362 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca006010-9836-457f-91cd-9ccd503decc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.298812 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-ca006010-9836-457f-91cd-9ccd503decc0 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:15.300056 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ca006010-9836-457f-91cd-9ccd503decc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:15.333888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1139/2264] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:15.339452 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12bb11-f51e-4218-b6ee-1da6b4109e71 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:15.358380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12bb11-f51e-4218-b6ee-1da6b4109e71 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:15.365149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd29950-ffea-4d24-a89a-ddbe8b5eb41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:15.366373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd29950-ffea-4d24-a89a-ddbe8b5eb41f admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.367702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd29950-ffea-4d24-a89a-ddbe8b5eb41f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.368079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd29950-ffea-4d24-a89a-ddbe8b5eb41f admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.368856 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd29950-ffea-4d24-a89a-ddbe8b5eb41f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:15.369284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd29950-ffea-4d24-a89a-ddbe8b5eb41f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:15.371711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd29950-ffea-4d24-a89a-ddbe8b5eb41f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:15.412733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1126/2265] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:15.416343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca006010-9836-457f-91cd-9ccd503decc0 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:15.429152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca006010-9836-457f-91cd-9ccd503decc0 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:15.446730 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764ea47-50b5-4441-b9f0-d361e2b5f2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:15.447355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764ea47-50b5-4441-b9f0-d361e2b5f2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.447637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764ea47-50b5-4441-b9f0-d361e2b5f2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.447908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764ea47-50b5-4441-b9f0-d361e2b5f2e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.448540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764ea47-50b5-4441-b9f0-d361e2b5f2e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:15.451016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764ea47-50b5-4441-b9f0-d361e2b5f2e2 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:15.577593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1140/2266] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:15 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:15.585828 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd29950-ffea-4d24-a89a-ddbe8b5eb41f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:15.600700 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd29950-ffea-4d24-a89a-ddbe8b5eb41f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:15.735133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b76f9-2e25-4461-89f2-a70fd41b31f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:15.737380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b76f9-2e25-4461-89f2-a70fd41b31f2 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.739315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b76f9-2e25-4461-89f2-a70fd41b31f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.739685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b76f9-2e25-4461-89f2-a70fd41b31f2 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.740497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b76f9-2e25-4461-89f2-a70fd41b31f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:15.743346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b76f9-2e25-4461-89f2-a70fd41b31f2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:15.798405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1127/2267] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:15 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:15.800142 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764ea47-50b5-4441-b9f0-d361e2b5f2e2 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:15.812344 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764ea47-50b5-4441-b9f0-d361e2b5f2e2 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:15.817783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e9d4a-7b8d-4dda-a160-619ee5ad1d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:15.818641 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e9d4a-7b8d-4dda-a160-619ee5ad1d38 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.818999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e9d4a-7b8d-4dda-a160-619ee5ad1d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.819448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e9d4a-7b8d-4dda-a160-619ee5ad1d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.820108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e9d4a-7b8d-4dda-a160-619ee5ad1d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:15.822219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e9d4a-7b8d-4dda-a160-619ee5ad1d38 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:15.840985 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1141/2268] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:15.842672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b76f9-2e25-4461-89f2-a70fd41b31f2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:15.856181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b76f9-2e25-4461-89f2-a70fd41b31f2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:15.867369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bb78c-a559-4b77-a096-b66b82ebe66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:15.868169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bb78c-a559-4b77-a096-b66b82ebe66d admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.868456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bb78c-a559-4b77-a096-b66b82ebe66d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.868766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bb78c-a559-4b77-a096-b66b82ebe66d admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.870734 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973bb78c-a559-4b77-a096-b66b82ebe66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:15.877953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973bb78c-a559-4b77-a096-b66b82ebe66d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:15.908167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1128/2269] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:15.912528 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e9d4a-7b8d-4dda-a160-619ee5ad1d38 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:15.930520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e9d4a-7b8d-4dda-a160-619ee5ad1d38 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:15.942802 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35371c11-c1ea-4715-8609-44445a00af5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:15.943598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35371c11-c1ea-4715-8609-44445a00af5d admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.943885 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35371c11-c1ea-4715-8609-44445a00af5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.944162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35371c11-c1ea-4715-8609-44445a00af5d admin admin] PATH_INFO: `/v3/projects/000248a86f34421092a269cdb05d9fb3/users/a031b5edff2f42aa942a8868925daabc/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.985196 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35371c11-c1ea-4715-8609-44445a00af5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=000248a86f34421092a269cdb05d9fb3, user_id=a031b5edff2f42aa942a8868925daabc, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:15.992577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35371c11-c1ea-4715-8609-44445a00af5d admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.029762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1142/2270] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:15 2022] PUT /identity/v3/projects/000248a86f34421092a269cdb05d9fb3/users/a031b5edff2f42aa942a8868925daabc/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:16.104098 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d9673-db3c-473d-95ec-e44aa0a67b1c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.104406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d9673-db3c-473d-95ec-e44aa0a67b1c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.104681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d9673-db3c-473d-95ec-e44aa0a67b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.159805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-868d9673-db3c-473d-95ec-e44aa0a67b1c None None] MFA Rules not processed for user `a031b5edff2f42aa942a8868925daabc`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:16.161464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-868d9673-db3c-473d-95ec-e44aa0a67b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:16.191562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:16.295423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35371c11-c1ea-4715-8609-44445a00af5d admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.322487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1129/2271] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:16.349596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:16.422538 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35371c11-c1ea-4715-8609-44445a00af5d admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.436085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d3c57-951a-4e63-8076-3d5cc48a515b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.436691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d3c57-951a-4e63-8076-3d5cc48a515b admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.437075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d3c57-951a-4e63-8076-3d5cc48a515b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.437231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d3c57-951a-4e63-8076-3d5cc48a515b admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.437987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d3c57-951a-4e63-8076-3d5cc48a515b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.440077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d3c57-951a-4e63-8076-3d5cc48a515b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.485950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d9673-db3c-473d-95ec-e44aa0a67b1c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.495377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1143/2272] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:16 2022] POST /identity/v3/projects => generated 377 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:16.503167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d9673-db3c-473d-95ec-e44aa0a67b1c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.619865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0976f8f-edfc-4104-bb8c-3d34e342d120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.620810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0976f8f-edfc-4104-bb8c-3d34e342d120 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.621313 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0976f8f-edfc-4104-bb8c-3d34e342d120 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.621927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0976f8f-edfc-4104-bb8c-3d34e342d120 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.635470 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0976f8f-edfc-4104-bb8c-3d34e342d120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.637807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0976f8f-edfc-4104-bb8c-3d34e342d120 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.699113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1130/2273] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:16 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:16.703315 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d3c57-951a-4e63-8076-3d5cc48a515b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.715622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d3c57-951a-4e63-8076-3d5cc48a515b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.730994 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e02c7b-8b04-49cc-aa78-abb2b6cb9eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.732824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e02c7b-8b04-49cc-aa78-abb2b6cb9eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.733790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e02c7b-8b04-49cc-aa78-abb2b6cb9eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.734855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e02c7b-8b04-49cc-aa78-abb2b6cb9eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.737012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e02c7b-8b04-49cc-aa78-abb2b6cb9eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.740646 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e02c7b-8b04-49cc-aa78-abb2b6cb9eb7 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.782035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0976f8f-edfc-4104-bb8c-3d34e342d120 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.793791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1144/2274] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:16.797573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0976f8f-edfc-4104-bb8c-3d34e342d120 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.811420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232d47ce-e227-4cdc-9c20-5b6e41a2b8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.812370 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d47ce-e227-4cdc-9c20-5b6e41a2b8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.812865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d47ce-e227-4cdc-9c20-5b6e41a2b8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.813141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232d47ce-e227-4cdc-9c20-5b6e41a2b8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.814014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d47ce-e227-4cdc-9c20-5b6e41a2b8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.815638 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232d47ce-e227-4cdc-9c20-5b6e41a2b8d1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.839390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1131/2275] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:16.843373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e02c7b-8b04-49cc-aa78-abb2b6cb9eb7 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:16.863229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e02c7b-8b04-49cc-aa78-abb2b6cb9eb7 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.883250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef98d5-c08e-46b8-baf8-d05889e5fb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.884343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef98d5-c08e-46b8-baf8-d05889e5fb59 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.884817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef98d5-c08e-46b8-baf8-d05889e5fb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.885297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef98d5-c08e-46b8-baf8-d05889e5fb59 admin admin] PATH_INFO: `/v3/projects/436a53eab146464184052bd750c1d01c/users/0df88efc53db47788e1a11526a03de43/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.906608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef98d5-c08e-46b8-baf8-d05889e5fb59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=436a53eab146464184052bd750c1d01c, user_id=0df88efc53db47788e1a11526a03de43, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.909837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfef98d5-c08e-46b8-baf8-d05889e5fb59 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.939718 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1145/2276] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:16 2022] PUT /identity/v3/projects/436a53eab146464184052bd750c1d01c/users/0df88efc53db47788e1a11526a03de43/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:16.951385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62daac7-70dc-465f-97dc-d191fdcc7b0b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.952012 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62daac7-70dc-465f-97dc-d191fdcc7b0b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.954154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62daac7-70dc-465f-97dc-d191fdcc7b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.048372 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b62daac7-70dc-465f-97dc-d191fdcc7b0b None None] MFA Rules not processed for user `0df88efc53db47788e1a11526a03de43`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:17.049710 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b62daac7-70dc-465f-97dc-d191fdcc7b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:17.207821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1132/2277] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:17.212534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef98d5-c08e-46b8-baf8-d05889e5fb59 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.373553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef98d5-c08e-46b8-baf8-d05889e5fb59 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.379581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9b7b2-53f8-4527-b76a-787f58b74633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.380822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9b7b2-53f8-4527-b76a-787f58b74633 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.381780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9b7b2-53f8-4527-b76a-787f58b74633 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.382071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9b7b2-53f8-4527-b76a-787f58b74633 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.382830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9b7b2-53f8-4527-b76a-787f58b74633 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.385530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9b7b2-53f8-4527-b76a-787f58b74633 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.475240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1146/2278] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:17 2022] POST /identity/v3/projects => generated 375 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:17.478565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62daac7-70dc-465f-97dc-d191fdcc7b0b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.514291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62daac7-70dc-465f-97dc-d191fdcc7b0b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.568435 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:17.655147 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be679d9-ee77-456e-8209-d2af477c97e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.670486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be679d9-ee77-456e-8209-d2af477c97e3 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.670486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be679d9-ee77-456e-8209-d2af477c97e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.670486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be679d9-ee77-456e-8209-d2af477c97e3 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.670486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be679d9-ee77-456e-8209-d2af477c97e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.670486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be679d9-ee77-456e-8209-d2af477c97e3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.672977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0c59-33ac-406c-bbb7-a3aad59107d9 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.673427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0c59-33ac-406c-bbb7-a3aad59107d9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.674262 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d0c59-33ac-406c-bbb7-a3aad59107d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.722298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1133/2279] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:17 2022] POST /identity/v3/users => generated 294 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:17.723170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be679d9-ee77-456e-8209-d2af477c97e3 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.731365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1e0d0c59-33ac-406c-bbb7-a3aad59107d9 None None] MFA Rules not processed for user `75642df28f5d4301b979f39482a9a11d`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:17.733167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1e0d0c59-33ac-406c-bbb7-a3aad59107d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:17.735519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be679d9-ee77-456e-8209-d2af477c97e3 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.746229 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27728392-0048-438b-80b5-9feb6d423a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.747401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27728392-0048-438b-80b5-9feb6d423a7f admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.747940 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27728392-0048-438b-80b5-9feb6d423a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.748426 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27728392-0048-438b-80b5-9feb6d423a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.749295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27728392-0048-438b-80b5-9feb6d423a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.751170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27728392-0048-438b-80b5-9feb6d423a7f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.765225 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1134/2280] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:17.784083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27728392-0048-438b-80b5-9feb6d423a7f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.799687 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27728392-0048-438b-80b5-9feb6d423a7f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:17.814531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9642ba2-0ffb-4d8d-bc57-f51acbbb4bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:17.815716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9642ba2-0ffb-4d8d-bc57-f51acbbb4bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:17.816668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9642ba2-0ffb-4d8d-bc57-f51acbbb4bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:17.819246 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9642ba2-0ffb-4d8d-bc57-f51acbbb4bb7 admin admin] PATH_INFO: `/v3/projects/7e78ceb8ba374c9889eb43083b3375f9/users/f1b3d7bb15754aaf9042dfc2fc5cb74a/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:17.842067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9642ba2-0ffb-4d8d-bc57-f51acbbb4bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e78ceb8ba374c9889eb43083b3375f9, user_id=f1b3d7bb15754aaf9042dfc2fc5cb74a, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:17.845878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9642ba2-0ffb-4d8d-bc57-f51acbbb4bb7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:17.861787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1147/2281] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:17.873337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1135/2282] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:17 2022] PUT /identity/v3/projects/7e78ceb8ba374c9889eb43083b3375f9/users/f1b3d7bb15754aaf9042dfc2fc5cb74a/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:17.880735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d0c59-33ac-406c-bbb7-a3aad59107d9 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:17.889384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:17.902950 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0d0c59-33ac-406c-bbb7-a3aad59107d9 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:18.085706 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8bf072-7475-4e0c-9693-81d926c46e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:18.086380 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bf072-7475-4e0c-9693-81d926c46e20 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:18.086658 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bf072-7475-4e0c-9693-81d926c46e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:18.087195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8bf072-7475-4e0c-9693-81d926c46e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:18.087864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bf072-7475-4e0c-9693-81d926c46e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:18.089484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8bf072-7475-4e0c-9693-81d926c46e20 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:18.101810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1148/2283] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:17 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:18.105287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9642ba2-0ffb-4d8d-bc57-f51acbbb4bb7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:18.116705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9642ba2-0ffb-4d8d-bc57-f51acbbb4bb7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:18.124380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a139c-71eb-404b-bd7e-0160c4e00289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:18.125151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a139c-71eb-404b-bd7e-0160c4e00289 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:18.125487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a139c-71eb-404b-bd7e-0160c4e00289 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:18.125828 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a139c-71eb-404b-bd7e-0160c4e00289 admin admin] PATH_INFO: `/v3/projects/7e78ceb8ba374c9889eb43083b3375f9/users/f1b3d7bb15754aaf9042dfc2fc5cb74a/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:18.129264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a139c-71eb-404b-bd7e-0160c4e00289 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e78ceb8ba374c9889eb43083b3375f9, user_id=f1b3d7bb15754aaf9042dfc2fc5cb74a, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:18.142610 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a139c-71eb-404b-bd7e-0160c4e00289 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:18.166835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1136/2284] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:18 2022] PUT /identity/v3/projects/7e78ceb8ba374c9889eb43083b3375f9/users/f1b3d7bb15754aaf9042dfc2fc5cb74a/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:18.173881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7982a9-05a9-469b-b3ea-1a0da7c539ff None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:18.174170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7982a9-05a9-469b-b3ea-1a0da7c539ff None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:18.174449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7982a9-05a9-469b-b3ea-1a0da7c539ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:18.211329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4e7982a9-05a9-469b-b3ea-1a0da7c539ff None None] MFA Rules not processed for user `f1b3d7bb15754aaf9042dfc2fc5cb74a`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:18.212935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4e7982a9-05a9-469b-b3ea-1a0da7c539ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:18.303541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1149/2285] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:18.317771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a139c-71eb-404b-bd7e-0160c4e00289 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:18.320467 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d19a139c-71eb-404b-bd7e-0160c4e00289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:18.391814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a139c-71eb-404b-bd7e-0160c4e00289 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:18.395548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b3d6b-9610-405f-aacd-ea2aaca5f2a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:18.396163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3d6b-9610-405f-aacd-ea2aaca5f2a3 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:18.396444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3d6b-9610-405f-aacd-ea2aaca5f2a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:18.396718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b3d6b-9610-405f-aacd-ea2aaca5f2a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:18.472159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b3d6b-9610-405f-aacd-ea2aaca5f2a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:18.473894 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b3d6b-9610-405f-aacd-ea2aaca5f2a3 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:18.484142 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1137/2286] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:20.176999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7982a9-05a9-469b-b3ea-1a0da7c539ff None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:20.178484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4e7982a9-05a9-469b-b3ea-1a0da7c539ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:20.189932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7982a9-05a9-469b-b3ea-1a0da7c539ff None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:20.193432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b28284-af0f-440c-9903-20746a945b79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:20.194062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b28284-af0f-440c-9903-20746a945b79 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:20.194340 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b28284-af0f-440c-9903-20746a945b79 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:20.194604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b28284-af0f-440c-9903-20746a945b79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:20.252770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b28284-af0f-440c-9903-20746a945b79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:20.254249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b28284-af0f-440c-9903-20746a945b79 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:20.263088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1150/2287] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:20.815695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b3d6b-9610-405f-aacd-ea2aaca5f2a3 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:20.824582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b3d6b-9610-405f-aacd-ea2aaca5f2a3 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:20.828300 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da216129-d791-4a94-aed3-40f30f2669ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:20.828906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da216129-d791-4a94-aed3-40f30f2669ea service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:20.829184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da216129-d791-4a94-aed3-40f30f2669ea service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:20.829781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da216129-d791-4a94-aed3-40f30f2669ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:20.838729 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da216129-d791-4a94-aed3-40f30f2669ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:20.840147 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da216129-d791-4a94-aed3-40f30f2669ea service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:20.850138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1138/2288] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:24.862678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b28284-af0f-440c-9903-20746a945b79 service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:24.891264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:24.934726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b28284-af0f-440c-9903-20746a945b79 service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:24.940488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:24.941135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:24.941490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:24.941794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] PATH_INFO: `/v3/users/a031b5edff2f42aa942a8868925daabc` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:24.942951 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a031b5edff2f42aa942a8868925daabc)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:24.944961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:25.000737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a031b5edff2f42aa942a8868925daabc'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.008005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a031b5edff2f42aa942a8868925daabc'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.008670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a031b5edff2f42aa942a8868925daabc'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.009281 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a031b5edff2f42aa942a8868925daabc'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.016567 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a031b5edff2f42aa942a8868925daabc'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.035924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1151/2289] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:24 2022] DELETE /identity/v3/users/a031b5edff2f42aa942a8868925daabc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:25.040406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f47f0-97df-4742-8ba1-aa582261c69b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:25.041307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f47f0-97df-4742-8ba1-aa582261c69b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:25.041589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f47f0-97df-4742-8ba1-aa582261c69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:25.061840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-289f47f0-97df-4742-8ba1-aa582261c69b None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:25.063130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-289f47f0-97df-4742-8ba1-aa582261c69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:25.071396 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:25.152335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1139/2290] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:25.162057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:25.163480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:25.223007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8b3ea-8077-4344-a636-fcad67351038 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:25.228200 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c471fa-2cd9-474d-9fb4-5f8f4d54701f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:25.228813 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c471fa-2cd9-474d-9fb4-5f8f4d54701f service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:25.229095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c471fa-2cd9-474d-9fb4-5f8f4d54701f service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:25.229368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c471fa-2cd9-474d-9fb4-5f8f4d54701f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:25.238865 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c471fa-2cd9-474d-9fb4-5f8f4d54701f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:25.240293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c471fa-2cd9-474d-9fb4-5f8f4d54701f service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:25.250668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1152/2291] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:25.561792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f47f0-97df-4742-8ba1-aa582261c69b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:25.563590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-289f47f0-97df-4742-8ba1-aa582261c69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:25.579988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f47f0-97df-4742-8ba1-aa582261c69b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:25.585543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:25.586576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:25.587384 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:25.587856 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] PATH_INFO: `/v3/projects/000248a86f34421092a269cdb05d9fb3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:25.589248 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=000248a86f34421092a269cdb05d9fb3)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:25.591587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:25.685838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '000248a86f34421092a269cdb05d9fb3', 'request_id': 'req-25cdd31f-95aa-410f-abdc-01053ed626bd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.692028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '000248a86f34421092a269cdb05d9fb3', 'request_id': 'req-25cdd31f-95aa-410f-abdc-01053ed626bd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.699681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '000248a86f34421092a269cdb05d9fb3', 'request_id': 'req-25cdd31f-95aa-410f-abdc-01053ed626bd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.713861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] The token cache is being invalidate because project 000248a86f34421092a269cdb05d9fb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:25.714159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.714816 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.716822 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1140/2292] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:25 2022] DELETE /identity/v3/projects/000248a86f34421092a269cdb05d9fb3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:25.720620 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c471fa-2cd9-474d-9fb4-5f8f4d54701f service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:25.732523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c471fa-2cd9-474d-9fb4-5f8f4d54701f service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:25.824456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:25.843181 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:25.843833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:25.844189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:25.844438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] PATH_INFO: `/v3/users/0df88efc53db47788e1a11526a03de43` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:25.845892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0df88efc53db47788e1a11526a03de43)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:25.847822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:25.903219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0df88efc53db47788e1a11526a03de43'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.917099 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0df88efc53db47788e1a11526a03de43'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.918060 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0df88efc53db47788e1a11526a03de43'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.918855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0df88efc53db47788e1a11526a03de43'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.926527 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0df88efc53db47788e1a11526a03de43'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:25.945822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1153/2293] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:25 2022] DELETE /identity/v3/users/0df88efc53db47788e1a11526a03de43 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:26.266001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:26.267485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:26.280070 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdd31f-95aa-410f-abdc-01053ed626bd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:26.339736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:26.340402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:26.340682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:26.340960 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] PATH_INFO: `/v3/projects/436a53eab146464184052bd750c1d01c` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:26.342631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=436a53eab146464184052bd750c1d01c)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:26.344568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:26.384743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:26.393545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '436a53eab146464184052bd750c1d01c', 'request_id': 'req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.400294 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '436a53eab146464184052bd750c1d01c', 'request_id': 'req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.406408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '436a53eab146464184052bd750c1d01c', 'request_id': 'req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.418482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] The token cache is being invalidate because project 436a53eab146464184052bd750c1d01c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:26.418772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.419388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.421390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1141/2294] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:26 2022] DELETE /identity/v3/projects/436a53eab146464184052bd750c1d01c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:26.425504 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:26.426960 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:26.440392 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff20afe-d769-4a33-abbe-2a5ad7cb0210 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:26.496634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a0cb9-7883-46a9-b82f-94f2029ffe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:26.497263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a0cb9-7883-46a9-b82f-94f2029ffe00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:26.497545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a0cb9-7883-46a9-b82f-94f2029ffe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:26.497858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a0cb9-7883-46a9-b82f-94f2029ffe00 admin admin] PATH_INFO: `/v3/users/f1b3d7bb15754aaf9042dfc2fc5cb74a` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:26.498984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a0cb9-7883-46a9-b82f-94f2029ffe00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b3d7bb15754aaf9042dfc2fc5cb74a)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:26.500899 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a0cb9-7883-46a9-b82f-94f2029ffe00 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:26.556002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-316a0cb9-7883-46a9-b82f-94f2029ffe00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b3d7bb15754aaf9042dfc2fc5cb74a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.564724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-316a0cb9-7883-46a9-b82f-94f2029ffe00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b3d7bb15754aaf9042dfc2fc5cb74a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.565364 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-316a0cb9-7883-46a9-b82f-94f2029ffe00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b3d7bb15754aaf9042dfc2fc5cb74a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.566750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-316a0cb9-7883-46a9-b82f-94f2029ffe00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b3d7bb15754aaf9042dfc2fc5cb74a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.575457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-316a0cb9-7883-46a9-b82f-94f2029ffe00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b3d7bb15754aaf9042dfc2fc5cb74a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:26.581002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:26.596970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1154/2295] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:26 2022] DELETE /identity/v3/users/f1b3d7bb15754aaf9042dfc2fc5cb74a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:26.928837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:26.928837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:26.943448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5719de96-b12a-40b1-a4e7-b25a3cdb6fe8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.008932 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54af61e-9397-48ce-b574-ba8d1a652a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.009705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54af61e-9397-48ce-b574-ba8d1a652a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.010020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54af61e-9397-48ce-b574-ba8d1a652a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.010291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54af61e-9397-48ce-b574-ba8d1a652a74 admin admin] PATH_INFO: `/v3/projects/7e78ceb8ba374c9889eb43083b3375f9` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.011421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54af61e-9397-48ce-b574-ba8d1a652a74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e78ceb8ba374c9889eb43083b3375f9)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.013482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54af61e-9397-48ce-b574-ba8d1a652a74 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.074656 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f54af61e-9397-48ce-b574-ba8d1a652a74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e78ceb8ba374c9889eb43083b3375f9', 'request_id': 'req-f54af61e-9397-48ce-b574-ba8d1a652a74'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.081421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f54af61e-9397-48ce-b574-ba8d1a652a74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e78ceb8ba374c9889eb43083b3375f9', 'request_id': 'req-f54af61e-9397-48ce-b574-ba8d1a652a74'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.087680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f54af61e-9397-48ce-b574-ba8d1a652a74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e78ceb8ba374c9889eb43083b3375f9', 'request_id': 'req-f54af61e-9397-48ce-b574-ba8d1a652a74'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.101935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f54af61e-9397-48ce-b574-ba8d1a652a74 admin admin] The token cache is being invalidate because project 7e78ceb8ba374c9889eb43083b3375f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:27.102225 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f54af61e-9397-48ce-b574-ba8d1a652a74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.102848 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f54af61e-9397-48ce-b574-ba8d1a652a74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:27.104333 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1142/2296] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:26 2022] DELETE /identity/v3/projects/7e78ceb8ba374c9889eb43083b3375f9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:27.109795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6063c2f-085b-41dc-bbf9-56ebed44b60d None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.110283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6063c2f-085b-41dc-bbf9-56ebed44b60d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.110370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6063c2f-085b-41dc-bbf9-56ebed44b60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.155010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b6063c2f-085b-41dc-bbf9-56ebed44b60d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:27.156571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b6063c2f-085b-41dc-bbf9-56ebed44b60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:27.231765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1155/2297] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:27.240205 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7021d7-dbbc-441b-ace2-b79c9f9c66e1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.240492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7021d7-dbbc-441b-ace2-b79c9f9c66e1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.240754 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7021d7-dbbc-441b-ace2-b79c9f9c66e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.270578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-1e7021d7-dbbc-441b-ace2-b79c9f9c66e1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:27.272513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1e7021d7-dbbc-441b-ace2-b79c9f9c66e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:27.285603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1143/2298] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:27.289549 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6063c2f-085b-41dc-bbf9-56ebed44b60d None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:27.303815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6063c2f-085b-41dc-bbf9-56ebed44b60d None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.311290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f56226-fd9f-4f4b-a718-408043fdf600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.312534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f56226-fd9f-4f4b-a718-408043fdf600 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.313079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f56226-fd9f-4f4b-a718-408043fdf600 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.313569 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f56226-fd9f-4f4b-a718-408043fdf600 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.314478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f56226-fd9f-4f4b-a718-408043fdf600 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:54:27.315038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f56226-fd9f-4f4b-a718-408043fdf600 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.316667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f56226-fd9f-4f4b-a718-408043fdf600 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.329864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1156/2299] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:54:27.333109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7021d7-dbbc-441b-ace2-b79c9f9c66e1 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:27.343696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7021d7-dbbc-441b-ace2-b79c9f9c66e1 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.348415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74c2f1-b8e0-491f-8fb3-c402262b508c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.349030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74c2f1-b8e0-491f-8fb3-c402262b508c admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.349306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74c2f1-b8e0-491f-8fb3-c402262b508c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.349575 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74c2f1-b8e0-491f-8fb3-c402262b508c admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.350287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74c2f1-b8e0-491f-8fb3-c402262b508c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.352194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74c2f1-b8e0-491f-8fb3-c402262b508c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.385829 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1144/2300] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:27 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:27.387333 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f56226-fd9f-4f4b-a718-408043fdf600 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:27.405762 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f56226-fd9f-4f4b-a718-408043fdf600 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.470463 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6aed9a-76df-4d56-a5c1-8aca6465dae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.471069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6aed9a-76df-4d56-a5c1-8aca6465dae1 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.471347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6aed9a-76df-4d56-a5c1-8aca6465dae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.471614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6aed9a-76df-4d56-a5c1-8aca6465dae1 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.472263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6aed9a-76df-4d56-a5c1-8aca6465dae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.474234 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6aed9a-76df-4d56-a5c1-8aca6465dae1 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.497288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1157/2301] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:27 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:27.501946 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74c2f1-b8e0-491f-8fb3-c402262b508c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:27.513679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74c2f1-b8e0-491f-8fb3-c402262b508c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.520037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6501c-2a66-41ec-8132-fa11170beb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.520769 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6501c-2a66-41ec-8132-fa11170beb2a admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.521133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6501c-2a66-41ec-8132-fa11170beb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.521473 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6501c-2a66-41ec-8132-fa11170beb2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.522231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6501c-2a66-41ec-8132-fa11170beb2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.524038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6501c-2a66-41ec-8132-fa11170beb2a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.538068 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1145/2302] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:27.539331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6aed9a-76df-4d56-a5c1-8aca6465dae1 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:27.550383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6aed9a-76df-4d56-a5c1-8aca6465dae1 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.555441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5e44b-8b7c-4f3a-9433-4e248342afa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.556065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5e44b-8b7c-4f3a-9433-4e248342afa1 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.556346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5e44b-8b7c-4f3a-9433-4e248342afa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.556619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5e44b-8b7c-4f3a-9433-4e248342afa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.557270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5e44b-8b7c-4f3a-9433-4e248342afa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.558961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5e44b-8b7c-4f3a-9433-4e248342afa1 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.570648 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1158/2303] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:27.574897 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6501c-2a66-41ec-8132-fa11170beb2a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:27.585003 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6501c-2a66-41ec-8132-fa11170beb2a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.590191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fa7e3-59a0-48dd-a230-e16563b86493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.590831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fa7e3-59a0-48dd-a230-e16563b86493 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.591105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fa7e3-59a0-48dd-a230-e16563b86493 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.591864 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55fa7e3-59a0-48dd-a230-e16563b86493 admin admin] PATH_INFO: `/v3/projects/e043585927d448e59ebd18173ccf1759/users/97ba8410efdc42ed884440dacac6dc0d/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.607909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fa7e3-59a0-48dd-a230-e16563b86493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e043585927d448e59ebd18173ccf1759, user_id=97ba8410efdc42ed884440dacac6dc0d, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.610948 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55fa7e3-59a0-48dd-a230-e16563b86493 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.613366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:27.626350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1146/2304] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:27 2022] PUT /identity/v3/projects/e043585927d448e59ebd18173ccf1759/users/97ba8410efdc42ed884440dacac6dc0d/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:27.636681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24db5de-0d72-401e-a51e-beea4727ef02 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.636977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24db5de-0d72-401e-a51e-beea4727ef02 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.637255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24db5de-0d72-401e-a51e-beea4727ef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.672892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-c24db5de-0d72-401e-a51e-beea4727ef02 None None] MFA Rules not processed for user `97ba8410efdc42ed884440dacac6dc0d`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:27.674460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c24db5de-0d72-401e-a51e-beea4727ef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:27.724195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:27.750403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1159/2305] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:27.755007 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fa7e3-59a0-48dd-a230-e16563b86493 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:27.817032 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55fa7e3-59a0-48dd-a230-e16563b86493 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.822015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4048c1b-e79d-4b26-92ac-f1b01bf42700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.822568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4048c1b-e79d-4b26-92ac-f1b01bf42700 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.822844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4048c1b-e79d-4b26-92ac-f1b01bf42700 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.823116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4048c1b-e79d-4b26-92ac-f1b01bf42700 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.823753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4048c1b-e79d-4b26-92ac-f1b01bf42700 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.825690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4048c1b-e79d-4b26-92ac-f1b01bf42700 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.852753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1147/2306] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:27 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:27.857589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24db5de-0d72-401e-a51e-beea4727ef02 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:27.868216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24db5de-0d72-401e-a51e-beea4727ef02 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.924532 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aca7b1-df2a-4d48-b91a-6ced2167a426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.925147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aca7b1-df2a-4d48-b91a-6ced2167a426 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.925429 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aca7b1-df2a-4d48-b91a-6ced2167a426 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.925703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aca7b1-df2a-4d48-b91a-6ced2167a426 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.926378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aca7b1-df2a-4d48-b91a-6ced2167a426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.928293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aca7b1-df2a-4d48-b91a-6ced2167a426 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.953353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1160/2307] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:27 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:27.954872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4048c1b-e79d-4b26-92ac-f1b01bf42700 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:27.968166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4048c1b-e79d-4b26-92ac-f1b01bf42700 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:27.973615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf49d9-41b9-4841-a987-51aa9bfd6656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:27.974247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf49d9-41b9-4841-a987-51aa9bfd6656 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:27.974531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf49d9-41b9-4841-a987-51aa9bfd6656 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:27.974808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf49d9-41b9-4841-a987-51aa9bfd6656 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:27.975456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf49d9-41b9-4841-a987-51aa9bfd6656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:27.977094 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf49d9-41b9-4841-a987-51aa9bfd6656 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:27.988016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1148/2308] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:54:27.992152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aca7b1-df2a-4d48-b91a-6ced2167a426 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.006244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aca7b1-df2a-4d48-b91a-6ced2167a426 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.010943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae21b4-81d6-4e79-8d69-af6d41825f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.011544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae21b4-81d6-4e79-8d69-af6d41825f59 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.011822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae21b4-81d6-4e79-8d69-af6d41825f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.012093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae21b4-81d6-4e79-8d69-af6d41825f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.012735 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae21b4-81d6-4e79-8d69-af6d41825f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.015048 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae21b4-81d6-4e79-8d69-af6d41825f59 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.028721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1161/2309] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:28.030439 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf49d9-41b9-4841-a987-51aa9bfd6656 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.041373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf49d9-41b9-4841-a987-51aa9bfd6656 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.046087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810d4b4-0d00-4b93-bce0-e1e43a4357e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.046738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810d4b4-0d00-4b93-bce0-e1e43a4357e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.047023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810d4b4-0d00-4b93-bce0-e1e43a4357e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.047299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9810d4b4-0d00-4b93-bce0-e1e43a4357e9 admin admin] PATH_INFO: `/v3/projects/08df768879f04bfb8a1b6390d2b0be3e/users/110b4a3618cb4a1d93ac905c983629a6/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.063356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810d4b4-0d00-4b93-bce0-e1e43a4357e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08df768879f04bfb8a1b6390d2b0be3e, user_id=110b4a3618cb4a1d93ac905c983629a6, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.066321 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9810d4b4-0d00-4b93-bce0-e1e43a4357e9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.079675 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1149/2310] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:28 2022] PUT /identity/v3/projects/08df768879f04bfb8a1b6390d2b0be3e/users/110b4a3618cb4a1d93ac905c983629a6/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:28.091054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01df156-7bc3-4edc-9941-ce243bd78d2c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.091342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01df156-7bc3-4edc-9941-ce243bd78d2c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.092127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01df156-7bc3-4edc-9941-ce243bd78d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.119877 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-f01df156-7bc3-4edc-9941-ce243bd78d2c None None] MFA Rules not processed for user `110b4a3618cb4a1d93ac905c983629a6`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:28.121053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f01df156-7bc3-4edc-9941-ce243bd78d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:28.182665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1162/2311] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:28.187602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810d4b4-0d00-4b93-bce0-e1e43a4357e9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.256344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9810d4b4-0d00-4b93-bce0-e1e43a4357e9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.262099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f14c5c-f589-41ff-b4fa-5013958bfb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.262875 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f14c5c-f589-41ff-b4fa-5013958bfb58 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.263263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f14c5c-f589-41ff-b4fa-5013958bfb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.263633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f14c5c-f589-41ff-b4fa-5013958bfb58 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.264391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f14c5c-f589-41ff-b4fa-5013958bfb58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.266392 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f14c5c-f589-41ff-b4fa-5013958bfb58 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.297528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1150/2312] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:54:28 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:54:28.301541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01df156-7bc3-4edc-9941-ce243bd78d2c None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.315483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01df156-7bc3-4edc-9941-ce243bd78d2c None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.370006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:28.384897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11a1c5-49d2-4695-99f6-db28ca9bc3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.385514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11a1c5-49d2-4695-99f6-db28ca9bc3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.385829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11a1c5-49d2-4695-99f6-db28ca9bc3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.386104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11a1c5-49d2-4695-99f6-db28ca9bc3c9 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.386749 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11a1c5-49d2-4695-99f6-db28ca9bc3c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.389208 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11a1c5-49d2-4695-99f6-db28ca9bc3c9 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.413652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1163/2313] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:54:28 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:54:28.415028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f14c5c-f589-41ff-b4fa-5013958bfb58 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.425313 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f14c5c-f589-41ff-b4fa-5013958bfb58 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.429913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e99ff3-4a57-4c99-9dcd-4bec928a91e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.430508 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e99ff3-4a57-4c99-9dcd-4bec928a91e1 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.430789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e99ff3-4a57-4c99-9dcd-4bec928a91e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.431060 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e99ff3-4a57-4c99-9dcd-4bec928a91e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.431704 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e99ff3-4a57-4c99-9dcd-4bec928a91e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.433287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e99ff3-4a57-4c99-9dcd-4bec928a91e1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.444092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1151/2314] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:28.448255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11a1c5-49d2-4695-99f6-db28ca9bc3c9 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.458753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11a1c5-49d2-4695-99f6-db28ca9bc3c9 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.463572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bcd9d-a8e4-4a88-8661-d00e08577690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.464216 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bcd9d-a8e4-4a88-8661-d00e08577690 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.464494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bcd9d-a8e4-4a88-8661-d00e08577690 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.464837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bcd9d-a8e4-4a88-8661-d00e08577690 admin admin] PATH_INFO: `/v3/projects/31b50cb56ea34b0085b5ac657c0669a8/users/5f59a4d21aad4693be0d275cfffd5581/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.481539 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bcd9d-a8e4-4a88-8661-d00e08577690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b50cb56ea34b0085b5ac657c0669a8, user_id=5f59a4d21aad4693be0d275cfffd5581, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.485074 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bcd9d-a8e4-4a88-8661-d00e08577690 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.498493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1164/2315] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:28 2022] PUT /identity/v3/projects/31b50cb56ea34b0085b5ac657c0669a8/users/5f59a4d21aad4693be0d275cfffd5581/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:28.502637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e99ff3-4a57-4c99-9dcd-4bec928a91e1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.512938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e99ff3-4a57-4c99-9dcd-4bec928a91e1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.525869 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:28.570763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef9631-337f-4097-b328-09654d2fd1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.571358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef9631-337f-4097-b328-09654d2fd1ca admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.571631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef9631-337f-4097-b328-09654d2fd1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.571900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef9631-337f-4097-b328-09654d2fd1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.572526 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef9631-337f-4097-b328-09654d2fd1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.574138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef9631-337f-4097-b328-09654d2fd1ca admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.583878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1152/2316] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:54:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:54:28.588262 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bcd9d-a8e4-4a88-8661-d00e08577690 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.598614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bcd9d-a8e4-4a88-8661-d00e08577690 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.603227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01885ed2-082a-43c5-9d10-b2154b073908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.603882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01885ed2-082a-43c5-9d10-b2154b073908 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.604158 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01885ed2-082a-43c5-9d10-b2154b073908 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.604430 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01885ed2-082a-43c5-9d10-b2154b073908 admin admin] PATH_INFO: `/v3/projects/31b50cb56ea34b0085b5ac657c0669a8/users/5f59a4d21aad4693be0d275cfffd5581/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.606392 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01885ed2-082a-43c5-9d10-b2154b073908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31b50cb56ea34b0085b5ac657c0669a8, user_id=5f59a4d21aad4693be0d275cfffd5581, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.609616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01885ed2-082a-43c5-9d10-b2154b073908 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.622207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1165/2317] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:54:28 2022] PUT /identity/v3/projects/31b50cb56ea34b0085b5ac657c0669a8/users/5f59a4d21aad4693be0d275cfffd5581/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:28.633016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e81a6-8dd8-40e9-b99f-291144f1932f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.633388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e81a6-8dd8-40e9-b99f-291144f1932f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.633802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e81a6-8dd8-40e9-b99f-291144f1932f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.671830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-024e81a6-8dd8-40e9-b99f-291144f1932f None None] MFA Rules not processed for user `5f59a4d21aad4693be0d275cfffd5581`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:28.673129 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-024e81a6-8dd8-40e9-b99f-291144f1932f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:28.755323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1153/2318] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:28.769364 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01885ed2-082a-43c5-9d10-b2154b073908 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:28.771300 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-01885ed2-082a-43c5-9d10-b2154b073908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:28.859459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01885ed2-082a-43c5-9d10-b2154b073908 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:28.863838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2e7a9-7dc3-40f4-820d-692b1c3bf17b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:28.865773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2e7a9-7dc3-40f4-820d-692b1c3bf17b service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.865773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2e7a9-7dc3-40f4-820d-692b1c3bf17b service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.865773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca2e7a9-7dc3-40f4-820d-692b1c3bf17b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.944481 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2e7a9-7dc3-40f4-820d-692b1c3bf17b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:28.946061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca2e7a9-7dc3-40f4-820d-692b1c3bf17b service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:28.956654 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1166/2319] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:30.541637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e81a6-8dd8-40e9-b99f-291144f1932f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:30.543116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-024e81a6-8dd8-40e9-b99f-291144f1932f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:30.554917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e81a6-8dd8-40e9-b99f-291144f1932f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:30.558505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecc753-d28e-4ba1-bef1-4ca57a39ba4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:30.559100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc753-d28e-4ba1-bef1-4ca57a39ba4d service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:30.559378 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc753-d28e-4ba1-bef1-4ca57a39ba4d service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:30.559647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecc753-d28e-4ba1-bef1-4ca57a39ba4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:30.629295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecc753-d28e-4ba1-bef1-4ca57a39ba4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:30.630880 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecc753-d28e-4ba1-bef1-4ca57a39ba4d service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:30.639979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1154/2320] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:31.253059 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2e7a9-7dc3-40f4-820d-692b1c3bf17b service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:31.262902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca2e7a9-7dc3-40f4-820d-692b1c3bf17b service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:31.267361 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac74d8-f27a-4d9a-907b-c93db81286e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:31.267974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac74d8-f27a-4d9a-907b-c93db81286e8 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:31.268254 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac74d8-f27a-4d9a-907b-c93db81286e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:31.268523 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac74d8-f27a-4d9a-907b-c93db81286e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:31.279603 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac74d8-f27a-4d9a-907b-c93db81286e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:31.281042 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac74d8-f27a-4d9a-907b-c93db81286e8 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:31.292369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1167/2321] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:35.359611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecc753-d28e-4ba1-bef1-4ca57a39ba4d service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:35.421243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:35.443621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecc753-d28e-4ba1-bef1-4ca57a39ba4d service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:35.448274 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:35.448913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.449195 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.449472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] PATH_INFO: `/v3/users/97ba8410efdc42ed884440dacac6dc0d` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.451196 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=97ba8410efdc42ed884440dacac6dc0d)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:35.454725 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:35.534677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97ba8410efdc42ed884440dacac6dc0d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:35.545397 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97ba8410efdc42ed884440dacac6dc0d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:35.546336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97ba8410efdc42ed884440dacac6dc0d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:35.547291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97ba8410efdc42ed884440dacac6dc0d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:35.555617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97ba8410efdc42ed884440dacac6dc0d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:35.579159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1155/2322] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:35 2022] DELETE /identity/v3/users/97ba8410efdc42ed884440dacac6dc0d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:35.583438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0fab1-eb4f-4187-8556-8b6a6045d375 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.583918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0fab1-eb4f-4187-8556-8b6a6045d375 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.584015 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0fab1-eb4f-4187-8556-8b6a6045d375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.603682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-8cf0fab1-eb4f-4187-8556-8b6a6045d375 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:35.604900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8cf0fab1-eb4f-4187-8556-8b6a6045d375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:35.643186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:35.667849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1168/2323] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:35.678171 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:35.679606 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:35.740109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b450b78-c0ee-41ed-aa20-b943d1e67c1f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:35.743929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c9346-5115-44b5-8904-c9a1f4cd292e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:35.744546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c9346-5115-44b5-8904-c9a1f4cd292e service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:35.744823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c9346-5115-44b5-8904-c9a1f4cd292e service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:35.745086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c9346-5115-44b5-8904-c9a1f4cd292e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:35.755108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c9346-5115-44b5-8904-c9a1f4cd292e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:35.756635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c9346-5115-44b5-8904-c9a1f4cd292e service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:35.767616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1156/2324] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:36.100335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0fab1-eb4f-4187-8556-8b6a6045d375 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:36.101856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8cf0fab1-eb4f-4187-8556-8b6a6045d375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:36.121291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0fab1-eb4f-4187-8556-8b6a6045d375 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:36.128453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:36.129186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.129557 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.129776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] PATH_INFO: `/v3/projects/e043585927d448e59ebd18173ccf1759` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.131181 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e043585927d448e59ebd18173ccf1759)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:36.133013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:36.200390 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e043585927d448e59ebd18173ccf1759', 'request_id': 'req-18b7ede1-62ec-49f2-a426-25e291928316'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.207648 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e043585927d448e59ebd18173ccf1759', 'request_id': 'req-18b7ede1-62ec-49f2-a426-25e291928316'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.217407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e043585927d448e59ebd18173ccf1759', 'request_id': 'req-18b7ede1-62ec-49f2-a426-25e291928316'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.229243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] The token cache is being invalidate because project e043585927d448e59ebd18173ccf1759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:36.229540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.230259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.232240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1169/2325] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:36 2022] DELETE /identity/v3/projects/e043585927d448e59ebd18173ccf1759 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:36.236015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c9346-5115-44b5-8904-c9a1f4cd292e service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:36.246463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c9346-5115-44b5-8904-c9a1f4cd292e service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:36.304065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:36.304625 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.304904 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.305172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] PATH_INFO: `/v3/users/110b4a3618cb4a1d93ac905c983629a6` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.306320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=110b4a3618cb4a1d93ac905c983629a6)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:36.308230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:36.309655 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:36.356885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110b4a3618cb4a1d93ac905c983629a6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.364107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110b4a3618cb4a1d93ac905c983629a6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.364713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110b4a3618cb4a1d93ac905c983629a6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.365312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110b4a3618cb4a1d93ac905c983629a6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.372309 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110b4a3618cb4a1d93ac905c983629a6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.390090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1157/2326] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:36 2022] DELETE /identity/v3/users/110b4a3618cb4a1d93ac905c983629a6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:36.661121 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:36.662627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:36.675952 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7ede1-62ec-49f2-a426-25e291928316 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:36.734313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:36.734941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.735222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.735496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] PATH_INFO: `/v3/projects/08df768879f04bfb8a1b6390d2b0be3e` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.736611 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] RBAC: Authorizing `identity:delete_project(project_id=08df768879f04bfb8a1b6390d2b0be3e)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:36.738579 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:36.784535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08df768879f04bfb8a1b6390d2b0be3e', 'request_id': 'req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.790707 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08df768879f04bfb8a1b6390d2b0be3e', 'request_id': 'req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.798743 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08df768879f04bfb8a1b6390d2b0be3e', 'request_id': 'req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.803870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:36.811514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] The token cache is being invalidate because project 08df768879f04bfb8a1b6390d2b0be3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:36.812038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.813044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:36.814646 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1170/2327] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:36 2022] DELETE /identity/v3/projects/08df768879f04bfb8a1b6390d2b0be3e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:36.818691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:36.820201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:36.834032 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa374a5-e998-4fde-bc6f-9ba8351fcb2f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:36.927788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:36.927788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:36.927788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:36.927788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] PATH_INFO: `/v3/users/5f59a4d21aad4693be0d275cfffd5581` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:36.928538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f59a4d21aad4693be0d275cfffd5581)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:36.931220 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:37.026958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f59a4d21aad4693be0d275cfffd5581'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:37.040356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f59a4d21aad4693be0d275cfffd5581'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:37.043578 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f59a4d21aad4693be0d275cfffd5581'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:37.044431 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f59a4d21aad4693be0d275cfffd5581'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:37.053081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f59a4d21aad4693be0d275cfffd5581'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:37.067056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:37.082474 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1158/2328] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:54:36 2022] DELETE /identity/v3/users/5f59a4d21aad4693be0d275cfffd5581 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:54:37.414814 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:37.416342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:37.443224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc9c09-b556-45fb-86ae-ffdeacb1c99e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:37.514676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:37.515463 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:37.515870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:37.516171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3 admin admin] PATH_INFO: `/v3/projects/31b50cb56ea34b0085b5ac657c0669a8` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:37.517701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31b50cb56ea34b0085b5ac657c0669a8)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:37.520049 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:37.581793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31b50cb56ea34b0085b5ac657c0669a8', 'request_id': 'req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:37.589186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31b50cb56ea34b0085b5ac657c0669a8', 'request_id': 'req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:37.597106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31b50cb56ea34b0085b5ac657c0669a8', 'request_id': 'req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:37.607986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3 admin admin] The token cache is being invalidate because project 31b50cb56ea34b0085b5ac657c0669a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:54:37.608277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:37.608895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-f8888e8f-f8d6-41c0-8d6c-1483f2b3fbe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:54:37.610408 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1171/2329] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:54:37 2022] DELETE /identity/v3/projects/31b50cb56ea34b0085b5ac657c0669a8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:54:44.793686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:44.795275 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:44.880739 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e353eb9-b48e-4bf2-807f-f848b34b5c92 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:44.935002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6a0fa-e5c1-4433-b18d-b4380626ecca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:44.936139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6a0fa-e5c1-4433-b18d-b4380626ecca service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:44.936416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6a0fa-e5c1-4433-b18d-b4380626ecca service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:44.936712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6a0fa-e5c1-4433-b18d-b4380626ecca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:44.938565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e0b6a0fa-e5c1-4433-b18d-b4380626ecca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:45.003858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6a0fa-e5c1-4433-b18d-b4380626ecca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:45.005284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6a0fa-e5c1-4433-b18d-b4380626ecca service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:45.016100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1159/2330] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:47.604722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:53.627383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1138657-c357-4d30-920c-d6831b985809 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:53.628103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1138657-c357-4d30-920c-d6831b985809 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:53.628103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1138657-c357-4d30-920c-d6831b985809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:53.647291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-e1138657-c357-4d30-920c-d6831b985809 None None] MFA Rules not processed for user `2e48a8c1b91142cf9d1af17f28e41961`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:53.648499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e1138657-c357-4d30-920c-d6831b985809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:53.720750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1172/2331] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:54:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:55.010635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=71 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:03.713349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=25 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:14.671588 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90a98-17e6-4d2c-8f10-5ed9dd61efe5 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:14.672130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90a98-17e6-4d2c-8f10-5ed9dd61efe5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:14.672130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a90a98-17e6-4d2c-8f10-5ed9dd61efe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:14.691635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c4a90a98-17e6-4d2c-8f10-5ed9dd61efe5 None None] MFA Rules not processed for user `9f07158e4fd1430e839a1592e1e11b58`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:14.692844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c4a90a98-17e6-4d2c-8f10-5ed9dd61efe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:14.750977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1160/2332] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:55:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:24.743082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=25 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:37.962833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1138657-c357-4d30-920c-d6831b985809 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:37.964529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e1138657-c357-4d30-920c-d6831b985809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:38.036269 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1138657-c357-4d30-920c-d6831b985809 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:38.043150 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216340e5-8575-4d42-a97c-c203920c4218 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:38.043822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216340e5-8575-4d42-a97c-c203920c4218 service placement] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:38.044102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216340e5-8575-4d42-a97c-c203920c4218 service placement] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:38.044480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216340e5-8575-4d42-a97c-c203920c4218 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:38.046185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-216340e5-8575-4d42-a97c-c203920c4218 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:38.059551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216340e5-8575-4d42-a97c-c203920c4218 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:38.061107 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216340e5-8575-4d42-a97c-c203920c4218 service placement] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:38.081162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1173/2333] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:48.074288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=32 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:00.912264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a363c49-50f7-4e0b-be16-c64d5eb512dc None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:00.912728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a363c49-50f7-4e0b-be16-c64d5eb512dc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:00.912831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a363c49-50f7-4e0b-be16-c64d5eb512dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:00.931599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9a363c49-50f7-4e0b-be16-c64d5eb512dc None None] MFA Rules not processed for user `e93b5304ecec4ceca33da4b58e8e2480`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:00.932778 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9a363c49-50f7-4e0b-be16-c64d5eb512dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:00.986445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1161/2334] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:10.979881 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=25 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:49.933271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24da2a-815a-4959-bd89-df5238cbbbf3 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:49.933828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24da2a-815a-4959-bd89-df5238cbbbf3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:49.933935 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24da2a-815a-4959-bd89-df5238cbbbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:49.959610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1c24da2a-815a-4959-bd89-df5238cbbbf3 None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:49.961054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1c24da2a-815a-4959-bd89-df5238cbbbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:50.026753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1174/2335] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 13:56:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:50.079323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a363c49-50f7-4e0b-be16-c64d5eb512dc None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:50.080765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9a363c49-50f7-4e0b-be16-c64d5eb512dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:50.154525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a363c49-50f7-4e0b-be16-c64d5eb512dc None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:50.159421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43abd2-75b5-4d42-b898-f6f1d96dbb6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:50.160009 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43abd2-75b5-4d42-b898-f6f1d96dbb6f service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:50.160285 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43abd2-75b5-4d42-b898-f6f1d96dbb6f service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:50.160554 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43abd2-75b5-4d42-b898-f6f1d96dbb6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:50.169043 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43abd2-75b5-4d42-b898-f6f1d96dbb6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:50.171231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f43abd2-75b5-4d42-b898-f6f1d96dbb6f service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:50.179815 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1162/2336] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 13:56:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:57:00.018411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=25 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:00.176889 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=32 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:09.154350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf4053-95ae-4cec-bb82-fd63a9702238 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:09.154888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf4053-95ae-4cec-bb82-fd63a9702238 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:09.155414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf4053-95ae-4cec-bb82-fd63a9702238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:09.197313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-2faf4053-95ae-4cec-bb82-fd63a9702238 None None] MFA Rules not processed for user `2e48a8c1b91142cf9d1af17f28e41961`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:09.198861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2faf4053-95ae-4cec-bb82-fd63a9702238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:09.208678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1175/2337] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:57:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:09.826519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab71fa0-fe95-4aac-9dbb-bf4bc23e7128 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:09.827430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab71fa0-fe95-4aac-9dbb-bf4bc23e7128 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:09.828115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab71fa0-fe95-4aac-9dbb-bf4bc23e7128 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:09.829648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1163/2338] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 13:57:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:57:09.847535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6200694-0331-4f84-b895-4b221e4403e9 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:09.848055 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6200694-0331-4f84-b895-4b221e4403e9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:09.848663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6200694-0331-4f84-b895-4b221e4403e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:09.870504 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-f6200694-0331-4f84-b895-4b221e4403e9 None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:09.871708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f6200694-0331-4f84-b895-4b221e4403e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:09.952082 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1176/2339] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 13:57:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:09.966994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab71fa0-fe95-4aac-9dbb-bf4bc23e7128 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:09.969381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-1ab71fa0-fe95-4aac-9dbb-bf4bc23e7128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:10.040072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab71fa0-fe95-4aac-9dbb-bf4bc23e7128 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:10.052994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c7bbf-4f8e-4771-aff0-6cd36bed8d75 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e27f5b81c704dbebbd6a83b2713e237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:10.053720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c7bbf-4f8e-4771-aff0-6cd36bed8d75 service glance] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:10.054101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c7bbf-4f8e-4771-aff0-6cd36bed8d75 service glance] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:10.054478 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c7bbf-4f8e-4771-aff0-6cd36bed8d75 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:10.068853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c7bbf-4f8e-4771-aff0-6cd36bed8d75 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:10.070492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c7bbf-4f8e-4771-aff0-6cd36bed8d75 service glance] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:10.082374 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1164/2340] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:10.165699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e9815-0e6b-4845-a5a4-d76edf318701 None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:10.166296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e9815-0e6b-4845-a5a4-d76edf318701 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:10.166296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e9815-0e6b-4845-a5a4-d76edf318701 None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:10.167907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1177/2341] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 13:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:57:10.172789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b68c72-cf98-4eab-8bb5-cd8bd4877070 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:10.173159 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b68c72-cf98-4eab-8bb5-cd8bd4877070 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:10.173445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b68c72-cf98-4eab-8bb5-cd8bd4877070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:10.206113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-48b68c72-cf98-4eab-8bb5-cd8bd4877070 None None] MFA Rules not processed for user `531ebc79bf424e9b8762b2c7ec7fe8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:10.207674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-48b68c72-cf98-4eab-8bb5-cd8bd4877070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:10.215272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1165/2342] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 13:57:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:10.228233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e9815-0e6b-4845-a5a4-d76edf318701 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:10.230018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-297e9815-0e6b-4845-a5a4-d76edf318701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:10.243245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e9815-0e6b-4845-a5a4-d76edf318701 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:10.246944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b46fc-890f-446f-bf49-cb17f7984ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:10.248183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b46fc-890f-446f-bf49-cb17f7984ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:10.248459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b46fc-890f-446f-bf49-cb17f7984ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:10.248727 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b46fc-890f-446f-bf49-cb17f7984ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:10.255832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b46fc-890f-446f-bf49-cb17f7984ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:10.257232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b46fc-890f-446f-bf49-cb17f7984ec3 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:10.265677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1178/2343] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:10.357239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0f909-747e-4247-b61b-d465afdc23b4 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:10.357551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0f909-747e-4247-b61b-d465afdc23b4 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:10.357866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0f909-747e-4247-b61b-d465afdc23b4 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:10.359259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1166/2344] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 13:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:57:10.364261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798572d-db4e-4cdf-bc79-a8e882d270e8 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:10.364617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798572d-db4e-4cdf-bc79-a8e882d270e8 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:10.364949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798572d-db4e-4cdf-bc79-a8e882d270e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:10.384995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-c798572d-db4e-4cdf-bc79-a8e882d270e8 None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:10.386231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c798572d-db4e-4cdf-bc79-a8e882d270e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:10.398443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1179/2345] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 13:57:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:10.409063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0f909-747e-4247-b61b-d465afdc23b4 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:10.420765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0f909-747e-4247-b61b-d465afdc23b4 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:10.424462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0646f-0d05-455f-8acb-d76953b63b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:10.425124 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0646f-0d05-455f-8acb-d76953b63b5e service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:10.425392 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0646f-0d05-455f-8acb-d76953b63b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:10.425670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0646f-0d05-455f-8acb-d76953b63b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:10.436715 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0646f-0d05-455f-8acb-d76953b63b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:10.438278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0646f-0d05-455f-8acb-d76953b63b5e service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:10.448815 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1167/2346] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:15.440727 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798572d-db4e-4cdf-bc79-a8e882d270e8 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:15.442232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c798572d-db4e-4cdf-bc79-a8e882d270e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:15.509993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798572d-db4e-4cdf-bc79-a8e882d270e8 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:15.514638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea879e8-3400-418d-ab24-4f04bdd00350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:15.515271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea879e8-3400-418d-ab24-4f04bdd00350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:15.515543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea879e8-3400-418d-ab24-4f04bdd00350 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:15.515811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea879e8-3400-418d-ab24-4f04bdd00350 admin admin] PATH_INFO: `/v3/users/e93b5304ecec4ceca33da4b58e8e2480` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:15.516934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea879e8-3400-418d-ab24-4f04bdd00350 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e93b5304ecec4ceca33da4b58e8e2480)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:15.518875 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea879e8-3400-418d-ab24-4f04bdd00350 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:15.568702 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-aea879e8-3400-418d-ab24-4f04bdd00350 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e93b5304ecec4ceca33da4b58e8e2480'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:15.577234 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-aea879e8-3400-418d-ab24-4f04bdd00350 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e93b5304ecec4ceca33da4b58e8e2480'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:15.578008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-aea879e8-3400-418d-ab24-4f04bdd00350 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e93b5304ecec4ceca33da4b58e8e2480'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:15.578649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-aea879e8-3400-418d-ab24-4f04bdd00350 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e93b5304ecec4ceca33da4b58e8e2480'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:15.585952 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-aea879e8-3400-418d-ab24-4f04bdd00350 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e93b5304ecec4ceca33da4b58e8e2480'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:15.591253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:15.611171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1180/2347] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:57:15 2022] DELETE /identity/v3/users/e93b5304ecec4ceca33da4b58e8e2480 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:57:15.782718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0646f-0d05-455f-8acb-d76953b63b5e service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:15.797013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0646f-0d05-455f-8acb-d76953b63b5e service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:15.853781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c230cf-bd53-4566-9418-4f7069cbd5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:15.854413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c230cf-bd53-4566-9418-4f7069cbd5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:15.854691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c230cf-bd53-4566-9418-4f7069cbd5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:15.854958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c230cf-bd53-4566-9418-4f7069cbd5b0 admin admin] PATH_INFO: `/v3/projects/0375e7ef3fb04600b5d4095057658bc7` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:15.856078 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c230cf-bd53-4566-9418-4f7069cbd5b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0375e7ef3fb04600b5d4095057658bc7)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:15.858024 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c230cf-bd53-4566-9418-4f7069cbd5b0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:15.909639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73c230cf-bd53-4566-9418-4f7069cbd5b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0375e7ef3fb04600b5d4095057658bc7', 'request_id': 'req-73c230cf-bd53-4566-9418-4f7069cbd5b0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:15.916148 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73c230cf-bd53-4566-9418-4f7069cbd5b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0375e7ef3fb04600b5d4095057658bc7', 'request_id': 'req-73c230cf-bd53-4566-9418-4f7069cbd5b0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:15.921717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73c230cf-bd53-4566-9418-4f7069cbd5b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0375e7ef3fb04600b5d4095057658bc7', 'request_id': 'req-73c230cf-bd53-4566-9418-4f7069cbd5b0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:15.933227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73c230cf-bd53-4566-9418-4f7069cbd5b0 admin admin] The token cache is being invalidate because project 0375e7ef3fb04600b5d4095057658bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:57:15.933512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73c230cf-bd53-4566-9418-4f7069cbd5b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:15.934160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-73c230cf-bd53-4566-9418-4f7069cbd5b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:15.935652 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1168/2348] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:57:15 2022] DELETE /identity/v3/projects/0375e7ef3fb04600b5d4095057658bc7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:57:15.940450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468ed73-f694-4d6e-897d-4f84c7cbb6b2 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:15.941171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468ed73-f694-4d6e-897d-4f84c7cbb6b2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:15.941171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468ed73-f694-4d6e-897d-4f84c7cbb6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:15.960898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-7468ed73-f694-4d6e-897d-4f84c7cbb6b2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:15.962138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7468ed73-f694-4d6e-897d-4f84c7cbb6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:16.020653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1181/2349] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:16.028704 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334288a5-4ea6-42c8-8a88-d757f3c68e07 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:16.029490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334288a5-4ea6-42c8-8a88-d757f3c68e07 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:16.029790 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334288a5-4ea6-42c8-8a88-d757f3c68e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:16.049027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-334288a5-4ea6-42c8-8a88-d757f3c68e07 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:16.050244 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-334288a5-4ea6-42c8-8a88-d757f3c68e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:16.058118 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1169/2350] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:16.062775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468ed73-f694-4d6e-897d-4f84c7cbb6b2 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:16.073002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468ed73-f694-4d6e-897d-4f84c7cbb6b2 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:16.078226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd7f3f-a141-44f0-b155-d7dead7e6849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:16.078828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd7f3f-a141-44f0-b155-d7dead7e6849 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:16.079109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd7f3f-a141-44f0-b155-d7dead7e6849 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:16.079378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd7f3f-a141-44f0-b155-d7dead7e6849 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:16.080022 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd7f3f-a141-44f0-b155-d7dead7e6849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:57:16.080323 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd7f3f-a141-44f0-b155-d7dead7e6849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:16.081926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd7f3f-a141-44f0-b155-d7dead7e6849 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:16.097146 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1182/2351] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:57:16.098545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334288a5-4ea6-42c8-8a88-d757f3c68e07 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:16.107726 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334288a5-4ea6-42c8-8a88-d757f3c68e07 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:16.112436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591b4fe-657a-40ba-a53d-0253b7cca5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:16.113045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591b4fe-657a-40ba-a53d-0253b7cca5ff admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:16.113324 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591b4fe-657a-40ba-a53d-0253b7cca5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:16.113595 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591b4fe-657a-40ba-a53d-0253b7cca5ff admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:16.114270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591b4fe-657a-40ba-a53d-0253b7cca5ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:16.116177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591b4fe-657a-40ba-a53d-0253b7cca5ff admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:16.120859 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:16.150248 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1170/2352] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:57:16 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:57:16.154827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd7f3f-a141-44f0-b155-d7dead7e6849 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:16.166147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd7f3f-a141-44f0-b155-d7dead7e6849 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:16.227032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3737b84-ba10-4722-8848-c2f42c0a0c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:16.227032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3737b84-ba10-4722-8848-c2f42c0a0c9f admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:16.227032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3737b84-ba10-4722-8848-c2f42c0a0c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:16.227320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3737b84-ba10-4722-8848-c2f42c0a0c9f admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:16.227846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3737b84-ba10-4722-8848-c2f42c0a0c9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:16.229864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3737b84-ba10-4722-8848-c2f42c0a0c9f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:16.254056 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1183/2353] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:57:16 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:57:16.258349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591b4fe-657a-40ba-a53d-0253b7cca5ff admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:16.269467 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591b4fe-657a-40ba-a53d-0253b7cca5ff admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:16.274203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed987535-b450-461f-83b6-c022185f3f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:16.274807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed987535-b450-461f-83b6-c022185f3f86 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:16.275115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed987535-b450-461f-83b6-c022185f3f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:16.275385 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed987535-b450-461f-83b6-c022185f3f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:16.276027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed987535-b450-461f-83b6-c022185f3f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:16.277607 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed987535-b450-461f-83b6-c022185f3f86 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:16.287776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1171/2354] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:57:16.294856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3737b84-ba10-4722-8848-c2f42c0a0c9f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:16.304803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3737b84-ba10-4722-8848-c2f42c0a0c9f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:16.312085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1071d4-d4ee-48c6-b12e-190f07fb0233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:16.312709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1071d4-d4ee-48c6-b12e-190f07fb0233 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:16.312990 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1071d4-d4ee-48c6-b12e-190f07fb0233 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:16.313260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1071d4-d4ee-48c6-b12e-190f07fb0233 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:16.313944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1071d4-d4ee-48c6-b12e-190f07fb0233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:16.315835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1071d4-d4ee-48c6-b12e-190f07fb0233 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:16.333881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1184/2355] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:57:16.334761 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed987535-b450-461f-83b6-c022185f3f86 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:16.355678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed987535-b450-461f-83b6-c022185f3f86 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:16.358402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd4acf-1147-495c-b878-8012449985f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:16.359133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd4acf-1147-495c-b878-8012449985f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:16.359599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd4acf-1147-495c-b878-8012449985f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:16.359880 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd4acf-1147-495c-b878-8012449985f6 admin admin] PATH_INFO: `/v3/projects/19d114fd258a4306b4c0665df90ae4c7/users/76a9dc3ddb144e599a2856fce8dc339d/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:16.383286 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd4acf-1147-495c-b878-8012449985f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d114fd258a4306b4c0665df90ae4c7, user_id=76a9dc3ddb144e599a2856fce8dc339d, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:16.386085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd4acf-1147-495c-b878-8012449985f6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:16.405879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1172/2356] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:57:16 2022] PUT /identity/v3/projects/19d114fd258a4306b4c0665df90ae4c7/users/76a9dc3ddb144e599a2856fce8dc339d/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:57:16.407749 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17071a-8c95-43b6-ad5c-430bee615812 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:16.408048 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17071a-8c95-43b6-ad5c-430bee615812 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:16.408325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17071a-8c95-43b6-ad5c-430bee615812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:16.427635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-0d17071a-8c95-43b6-ad5c-430bee615812 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:16.428836 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0d17071a-8c95-43b6-ad5c-430bee615812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:16.476046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:16.492319 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1185/2357] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:16.502632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd4acf-1147-495c-b878-8012449985f6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:16.504331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-10bd4acf-1147-495c-b878-8012449985f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:16.562590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd4acf-1147-495c-b878-8012449985f6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:16.566201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34109cc-6b94-4b28-9728-54a76d68c30b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:16.566801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34109cc-6b94-4b28-9728-54a76d68c30b service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:16.567073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34109cc-6b94-4b28-9728-54a76d68c30b service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:16.567340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34109cc-6b94-4b28-9728-54a76d68c30b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:16.576509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34109cc-6b94-4b28-9728-54a76d68c30b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:16.577964 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34109cc-6b94-4b28-9728-54a76d68c30b service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:16.587518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1173/2358] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:21.057545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d62bb-d7ad-462c-8067-18f2e2f14321 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:21.058412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d62bb-d7ad-462c-8067-18f2e2f14321 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:21.058696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d62bb-d7ad-462c-8067-18f2e2f14321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:21.081441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4d0d62bb-d7ad-462c-8067-18f2e2f14321 None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:21.082700 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4d0d62bb-d7ad-462c-8067-18f2e2f14321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:21.146453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1186/2359] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 13:57:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:21.156566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34109cc-6b94-4b28-9728-54a76d68c30b service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:21.158255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a34109cc-6b94-4b28-9728-54a76d68c30b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:21.260015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34109cc-6b94-4b28-9728-54a76d68c30b service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:21.267333 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c987533-551b-4f95-b028-22393eb42c48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:21.267967 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c987533-551b-4f95-b028-22393eb42c48 service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:21.268251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c987533-551b-4f95-b028-22393eb42c48 service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:21.268712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c987533-551b-4f95-b028-22393eb42c48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:21.278175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c987533-551b-4f95-b028-22393eb42c48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:21.279931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c987533-551b-4f95-b028-22393eb42c48 service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:21.290209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1174/2360] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 13:57:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:57:23.598758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d62bb-d7ad-462c-8067-18f2e2f14321 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:23.600564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4d0d62bb-d7ad-462c-8067-18f2e2f14321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:23.624011 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d62bb-d7ad-462c-8067-18f2e2f14321 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:23.630419 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1539e-6e01-4ce9-99ba-a43c015537a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:23.631259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1539e-6e01-4ce9-99ba-a43c015537a6 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:23.631655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1539e-6e01-4ce9-99ba-a43c015537a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:23.633229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a1539e-6e01-4ce9-99ba-a43c015537a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:23.635270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-51a1539e-6e01-4ce9-99ba-a43c015537a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:23.788599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1539e-6e01-4ce9-99ba-a43c015537a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:23.790040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a1539e-6e01-4ce9-99ba-a43c015537a6 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:23.807894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1187/2361] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:24.819656 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87225b4-7948-4013-9f30-6fe15bd6ac6f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:24.820309 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87225b4-7948-4013-9f30-6fe15bd6ac6f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:24.820309 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87225b4-7948-4013-9f30-6fe15bd6ac6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:24.876314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c87225b4-7948-4013-9f30-6fe15bd6ac6f None None] MFA Rules not processed for user `76a9dc3ddb144e599a2856fce8dc339d`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:24.877666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c87225b4-7948-4013-9f30-6fe15bd6ac6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:24.938499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:24.996438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1175/2362] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:57:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:25.000871 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1539e-6e01-4ce9-99ba-a43c015537a6 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:25.005463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87225b4-7948-4013-9f30-6fe15bd6ac6f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:25.013487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a1539e-6e01-4ce9-99ba-a43c015537a6 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:25.018078 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87225b4-7948-4013-9f30-6fe15bd6ac6f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:25.018837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256578b9-a292-40d5-bd79-2ad84b40f21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:25.019541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256578b9-a292-40d5-bd79-2ad84b40f21c admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:25.019894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256578b9-a292-40d5-bd79-2ad84b40f21c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:25.020174 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256578b9-a292-40d5-bd79-2ad84b40f21c admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:25.020825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256578b9-a292-40d5-bd79-2ad84b40f21c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:25.022167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437641f-1f9f-4520-9335-aa8a725adaa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:25.022950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437641f-1f9f-4520-9335-aa8a725adaa0 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:25.023080 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256578b9-a292-40d5-bd79-2ad84b40f21c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:25.023369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437641f-1f9f-4520-9335-aa8a725adaa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:25.023709 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437641f-1f9f-4520-9335-aa8a725adaa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:25.025667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0437641f-1f9f-4520-9335-aa8a725adaa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:25.040788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437641f-1f9f-4520-9335-aa8a725adaa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:25.042269 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437641f-1f9f-4520-9335-aa8a725adaa0 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:25.057787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1176/2363] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:57:25.058805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1188/2364] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:57:24 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:57:25.063258 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437641f-1f9f-4520-9335-aa8a725adaa0 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:25.095488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437641f-1f9f-4520-9335-aa8a725adaa0 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:25.212250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16750621-fba6-4ada-a45a-3d474106e375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:25.212939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16750621-fba6-4ada-a45a-3d474106e375 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:25.213217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16750621-fba6-4ada-a45a-3d474106e375 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:25.213509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16750621-fba6-4ada-a45a-3d474106e375 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:25.214539 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16750621-fba6-4ada-a45a-3d474106e375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:25.223277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16750621-fba6-4ada-a45a-3d474106e375 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:25.258475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1177/2365] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:57:25 2022] POST /identity/v3/users => generated 296 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:57:25.261472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256578b9-a292-40d5-bd79-2ad84b40f21c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:25.272843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256578b9-a292-40d5-bd79-2ad84b40f21c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:25.287879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728594c-27bb-4447-a710-9920b201dc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:25.288508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728594c-27bb-4447-a710-9920b201dc20 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:25.288862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728594c-27bb-4447-a710-9920b201dc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:25.289138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728594c-27bb-4447-a710-9920b201dc20 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:25.289822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728594c-27bb-4447-a710-9920b201dc20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:25.291418 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728594c-27bb-4447-a710-9920b201dc20 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:25.323326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1189/2366] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:57:25.327430 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16750621-fba6-4ada-a45a-3d474106e375 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:25.346282 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16750621-fba6-4ada-a45a-3d474106e375 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:25.355348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6be28-3f4e-4700-ac98-81f6ed0becfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:25.356921 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6be28-3f4e-4700-ac98-81f6ed0becfa admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:25.368997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6be28-3f4e-4700-ac98-81f6ed0becfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:25.369602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6be28-3f4e-4700-ac98-81f6ed0becfa admin admin] PATH_INFO: `/v3/projects/bd737a2dedef4e1cb1ea490cd45a985d/users/b30ed315686e4801b490d5d658de5d7f/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:25.388913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6be28-3f4e-4700-ac98-81f6ed0becfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd737a2dedef4e1cb1ea490cd45a985d, user_id=b30ed315686e4801b490d5d658de5d7f, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:25.391802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6be28-3f4e-4700-ac98-81f6ed0becfa admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:25.409545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1178/2367] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:57:25 2022] PUT /identity/v3/projects/bd737a2dedef4e1cb1ea490cd45a985d/users/b30ed315686e4801b490d5d658de5d7f/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:57:25.414124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728594c-27bb-4447-a710-9920b201dc20 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:25.433113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728594c-27bb-4447-a710-9920b201dc20 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:25.505940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:25.524854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e84994-4342-4d60-83cb-aa7a8bfd2520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:25.525678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e84994-4342-4d60-83cb-aa7a8bfd2520 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:25.526253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e84994-4342-4d60-83cb-aa7a8bfd2520 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:25.526418 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e84994-4342-4d60-83cb-aa7a8bfd2520 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:25.529761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e84994-4342-4d60-83cb-aa7a8bfd2520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:25.529761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e84994-4342-4d60-83cb-aa7a8bfd2520 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:25.544322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1190/2368] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:57:25.546999 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6be28-3f4e-4700-ac98-81f6ed0becfa admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:25.558434 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6be28-3f4e-4700-ac98-81f6ed0becfa admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:25.574428 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842a8ba-27ac-4f15-8065-dd463e1b856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:25.575654 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842a8ba-27ac-4f15-8065-dd463e1b856e admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:25.576088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842a8ba-27ac-4f15-8065-dd463e1b856e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:25.576374 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842a8ba-27ac-4f15-8065-dd463e1b856e admin admin] PATH_INFO: `/v3/projects/bd737a2dedef4e1cb1ea490cd45a985d/users/b30ed315686e4801b490d5d658de5d7f/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:25.580965 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e842a8ba-27ac-4f15-8065-dd463e1b856e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd737a2dedef4e1cb1ea490cd45a985d, user_id=b30ed315686e4801b490d5d658de5d7f, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:25.583739 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e842a8ba-27ac-4f15-8065-dd463e1b856e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:25.603763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1179/2369] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:57:25 2022] PUT /identity/v3/projects/bd737a2dedef4e1cb1ea490cd45a985d/users/b30ed315686e4801b490d5d658de5d7f/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:57:32.516338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e98366-562f-4bd9-bfcf-011e76f96721 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:32.516878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e98366-562f-4bd9-bfcf-011e76f96721 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:32.516878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e98366-562f-4bd9-bfcf-011e76f96721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:32.550525 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-f1e98366-562f-4bd9-bfcf-011e76f96721 None None] MFA Rules not processed for user `b30ed315686e4801b490d5d658de5d7f`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:32.551757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f1e98366-562f-4bd9-bfcf-011e76f96721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:32.640780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1191/2370] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:57:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:32.654817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842a8ba-27ac-4f15-8065-dd463e1b856e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:32.656881 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e842a8ba-27ac-4f15-8065-dd463e1b856e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:32.745395 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e842a8ba-27ac-4f15-8065-dd463e1b856e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:32.751256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca3db7-c57c-41bf-a5a3-1afa5a2b48f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:32.752011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca3db7-c57c-41bf-a5a3-1afa5a2b48f1 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:32.752386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca3db7-c57c-41bf-a5a3-1afa5a2b48f1 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:32.752760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca3db7-c57c-41bf-a5a3-1afa5a2b48f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:32.763150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:32.819828 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca3db7-c57c-41bf-a5a3-1afa5a2b48f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:32.822030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca3db7-c57c-41bf-a5a3-1afa5a2b48f1 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:32.832509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1180/2371] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:42.344587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88c598-d28a-44a2-b26d-ae9142c40037 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:42.345129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88c598-d28a-44a2-b26d-ae9142c40037 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:42.345129 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88c598-d28a-44a2-b26d-ae9142c40037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:42.367048 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-fc88c598-d28a-44a2-b26d-ae9142c40037 None None] MFA Rules not processed for user `9f07158e4fd1430e839a1592e1e11b58`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:42.368352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-fc88c598-d28a-44a2-b26d-ae9142c40037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:42.426772 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1192/2372] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:57:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:42.827978 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=24 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:44.096574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca3db7-c57c-41bf-a5a3-1afa5a2b48f1 service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:44.098267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-75ca3db7-c57c-41bf-a5a3-1afa5a2b48f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:44.113258 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca3db7-c57c-41bf-a5a3-1afa5a2b48f1 service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:44.118841 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c9012-00fb-47e9-98a8-d9cc732c0bd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:44.119451 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c9012-00fb-47e9-98a8-d9cc732c0bd8 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:44.119730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c9012-00fb-47e9-98a8-d9cc732c0bd8 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:44.120006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c9012-00fb-47e9-98a8-d9cc732c0bd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:44.122293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e77c9012-00fb-47e9-98a8-d9cc732c0bd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:44.135927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c9012-00fb-47e9-98a8-d9cc732c0bd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:44.138130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c9012-00fb-47e9-98a8-d9cc732c0bd8 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:44.149723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1181/2373] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:47.443535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04940b-89c2-491d-af42-0c6274cce429 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:47.444178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04940b-89c2-491d-af42-0c6274cce429 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:47.444737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b04940b-89c2-491d-af42-0c6274cce429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:47.467218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4b04940b-89c2-491d-af42-0c6274cce429 None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:47.468489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4b04940b-89c2-491d-af42-0c6274cce429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:47.537345 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1193/2374] 10.222.0.55 () {58 vars in 967 bytes} [Thu Sep 8 13:57:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:47.549853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c9012-00fb-47e9-98a8-d9cc732c0bd8 service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:47.618340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c9012-00fb-47e9-98a8-d9cc732c0bd8 service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:47.624251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72c7f3-030b-4eaa-b7a7-6ad663f7af3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:47.624864 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72c7f3-030b-4eaa-b7a7-6ad663f7af3c service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:47.625143 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72c7f3-030b-4eaa-b7a7-6ad663f7af3c service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:47.625418 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72c7f3-030b-4eaa-b7a7-6ad663f7af3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:47.633392 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72c7f3-030b-4eaa-b7a7-6ad663f7af3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:47.635098 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72c7f3-030b-4eaa-b7a7-6ad663f7af3c service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:47.644527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1182/2375] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 13:57:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:57:54.660444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432fc62-90c5-48e6-910d-60cbe33991ff None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:54.660989 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432fc62-90c5-48e6-910d-60cbe33991ff None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:54.660989 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432fc62-90c5-48e6-910d-60cbe33991ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:54.681141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-8432fc62-90c5-48e6-910d-60cbe33991ff None None] MFA Rules not processed for user `76a9dc3ddb144e599a2856fce8dc339d`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:54.682386 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8432fc62-90c5-48e6-910d-60cbe33991ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:54.689822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1194/2376] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:57:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:55.022191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72c7f3-030b-4eaa-b7a7-6ad663f7af3c service swift] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:55.085767 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72c7f3-030b-4eaa-b7a7-6ad663f7af3c service swift] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:55.089555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01e537-06f8-41a8-aaf8-5a838439e752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:55.090701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01e537-06f8-41a8-aaf8-5a838439e752 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:55.090976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01e537-06f8-41a8-aaf8-5a838439e752 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:55.091248 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01e537-06f8-41a8-aaf8-5a838439e752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:55.092971 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8a01e537-06f8-41a8-aaf8-5a838439e752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:55.102209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01e537-06f8-41a8-aaf8-5a838439e752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:55.103624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01e537-06f8-41a8-aaf8-5a838439e752 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:55.112000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1183/2377] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:55.638304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432fc62-90c5-48e6-910d-60cbe33991ff None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:55.639874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8432fc62-90c5-48e6-910d-60cbe33991ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:55.645033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=95 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:55.724027 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432fc62-90c5-48e6-910d-60cbe33991ff None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:55.729382 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088426f5-9bbc-4af6-96c9-f15350a911f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:55.730022 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088426f5-9bbc-4af6-96c9-f15350a911f2 service placement] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:55.730303 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088426f5-9bbc-4af6-96c9-f15350a911f2 service placement] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:55.731240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088426f5-9bbc-4af6-96c9-f15350a911f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:55.733006 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-088426f5-9bbc-4af6-96c9-f15350a911f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:55.747007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088426f5-9bbc-4af6-96c9-f15350a911f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:55.748534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088426f5-9bbc-4af6-96c9-f15350a911f2 service placement] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:55.759175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1195/2378] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:58.199464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112029b-8884-43bc-87cd-7f49ccb161e1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:58.199961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112029b-8884-43bc-87cd-7f49ccb161e1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:58.200068 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112029b-8884-43bc-87cd-7f49ccb161e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:58.220408 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8112029b-8884-43bc-87cd-7f49ccb161e1 None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:58.221944 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8112029b-8884-43bc-87cd-7f49ccb161e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:58.232001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1184/2379] 10.222.0.55 () {58 vars in 967 bytes} [Thu Sep 8 13:57:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:58.254654 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088426f5-9bbc-4af6-96c9-f15350a911f2 service placement] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:58.267737 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088426f5-9bbc-4af6-96c9-f15350a911f2 service placement] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:58.273841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49116-227b-4500-9c95-8e57be181113 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:58.275128 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49116-227b-4500-9c95-8e57be181113 service swift] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:58.275634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49116-227b-4500-9c95-8e57be181113 service swift] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:58.276100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d49116-227b-4500-9c95-8e57be181113 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:58.287002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d49116-227b-4500-9c95-8e57be181113 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:58.289140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d49116-227b-4500-9c95-8e57be181113 service swift] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:58.298744 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1196/2380] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 13:57:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:57:59.149454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112029b-8884-43bc-87cd-7f49ccb161e1 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:59.163517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112029b-8884-43bc-87cd-7f49ccb161e1 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:59.171304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140654d-ecaa-4264-917e-6d0fda12817a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:59.172074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140654d-ecaa-4264-917e-6d0fda12817a service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:59.172427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140654d-ecaa-4264-917e-6d0fda12817a service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:59.172622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140654d-ecaa-4264-917e-6d0fda12817a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:59.174589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9140654d-ecaa-4264-917e-6d0fda12817a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:59.194672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140654d-ecaa-4264-917e-6d0fda12817a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:59.196343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140654d-ecaa-4264-917e-6d0fda12817a service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:59.207945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1185/2381] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:05.688647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49116-227b-4500-9c95-8e57be181113 service swift] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:05.709045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d49116-227b-4500-9c95-8e57be181113 service swift] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:05.715514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d7ec9-9a15-4803-9337-2117dc06a8ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:05.716603 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d7ec9-9a15-4803-9337-2117dc06a8ee service placement] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:05.716894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d7ec9-9a15-4803-9337-2117dc06a8ee service placement] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:05.717570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d7ec9-9a15-4803-9337-2117dc06a8ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:05.722627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-949d7ec9-9a15-4803-9337-2117dc06a8ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:05.739651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d7ec9-9a15-4803-9337-2117dc06a8ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:05.741248 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d7ec9-9a15-4803-9337-2117dc06a8ee service placement] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:05.752002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1197/2382] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:05.766509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140654d-ecaa-4264-917e-6d0fda12817a service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:05.769591 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9140654d-ecaa-4264-917e-6d0fda12817a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:05.842630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140654d-ecaa-4264-917e-6d0fda12817a service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:05.847208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:05.847780 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:05.848061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:05.848369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] PATH_INFO: `/v3/users/bed19e9a74be46b5bc836c3cd605133f` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:05.849608 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bed19e9a74be46b5bc836c3cd605133f)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:05.851714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:05.853212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:05.906980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bed19e9a74be46b5bc836c3cd605133f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:05.914753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bed19e9a74be46b5bc836c3cd605133f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:05.915433 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bed19e9a74be46b5bc836c3cd605133f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:05.916060 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bed19e9a74be46b5bc836c3cd605133f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:05.923677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bed19e9a74be46b5bc836c3cd605133f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:05.940605 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1186/2383] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:58:05 2022] DELETE /identity/v3/users/bed19e9a74be46b5bc836c3cd605133f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:58:06.109103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d7ec9-9a15-4803-9337-2117dc06a8ee service placement] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:06.123441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d7ec9-9a15-4803-9337-2117dc06a8ee service placement] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:06.182745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:06.183419 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.183696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.183963 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] PATH_INFO: `/v3/projects/f7507a7da3bb4420abcfcf08780dc0a4` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.185083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7507a7da3bb4420abcfcf08780dc0a4)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:06.187024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:06.233705 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7507a7da3bb4420abcfcf08780dc0a4', 'request_id': 'req-476d0c73-b487-42f6-a630-4a7ed330baed'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.239834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7507a7da3bb4420abcfcf08780dc0a4', 'request_id': 'req-476d0c73-b487-42f6-a630-4a7ed330baed'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.247157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7507a7da3bb4420abcfcf08780dc0a4', 'request_id': 'req-476d0c73-b487-42f6-a630-4a7ed330baed'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.252014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:06.259530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] The token cache is being invalidate because project f7507a7da3bb4420abcfcf08780dc0a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:58:06.260066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.261023 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.263036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1198/2384] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:58:06 2022] DELETE /identity/v3/projects/f7507a7da3bb4420abcfcf08780dc0a4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:58:06.266993 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:06.268429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:06.281977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d416ed7-090d-456f-8715-3ddd9483e5d4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:06.337152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:06.337802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.338085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.338356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] PATH_INFO: `/v3/users/5e67e78bbec9437988872c535aadb415` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.339808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e67e78bbec9437988872c535aadb415)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:06.342484 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:06.388440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e67e78bbec9437988872c535aadb415'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.395827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e67e78bbec9437988872c535aadb415'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.396453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e67e78bbec9437988872c535aadb415'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.397163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e67e78bbec9437988872c535aadb415'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.404462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e67e78bbec9437988872c535aadb415'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.417049 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:06.422936 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1187/2385] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:58:06 2022] DELETE /identity/v3/users/5e67e78bbec9437988872c535aadb415 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:58:06.607651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:06.609127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:06.622555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d0c73-b487-42f6-a630-4a7ed330baed admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:06.682028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:06.682666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:06.683118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:06.683898 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] PATH_INFO: `/v3/projects/444e5e44ba7f48348c36f5e2bf864df0` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:06.685035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=444e5e44ba7f48348c36f5e2bf864df0)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:06.687005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:06.733716 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444e5e44ba7f48348c36f5e2bf864df0', 'request_id': 'req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.739906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444e5e44ba7f48348c36f5e2bf864df0', 'request_id': 'req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.747664 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444e5e44ba7f48348c36f5e2bf864df0', 'request_id': 'req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.758145 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] The token cache is being invalidate because project 444e5e44ba7f48348c36f5e2bf864df0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:58:06.758436 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.759062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:58:06.760521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1199/2386] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:58:06 2022] DELETE /identity/v3/projects/444e5e44ba7f48348c36f5e2bf864df0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:58:13.905226 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:13.906703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:13.963603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93fefe-77b8-41ef-a50a-df5fc395ace1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:14.013641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695e711-6ed4-46f6-b19b-37f6f37367aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:14.014334 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695e711-6ed4-46f6-b19b-37f6f37367aa service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:14.014618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695e711-6ed4-46f6-b19b-37f6f37367aa service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:14.014891 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695e711-6ed4-46f6-b19b-37f6f37367aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:14.016618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b695e711-6ed4-46f6-b19b-37f6f37367aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:14.082195 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695e711-6ed4-46f6-b19b-37f6f37367aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:14.083555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695e711-6ed4-46f6-b19b-37f6f37367aa service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:14.093204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1188/2387] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:15.869134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:15.870793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:15.948941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda7264-c7a4-47d7-a205-b2c6b8f8c548 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:15.955794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020773e-9b26-4979-90ab-00ded3ca3205 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:15.956024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020773e-9b26-4979-90ab-00ded3ca3205 service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:15.956303 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020773e-9b26-4979-90ab-00ded3ca3205 service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:15.956577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0020773e-9b26-4979-90ab-00ded3ca3205 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:15.958367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0020773e-9b26-4979-90ab-00ded3ca3205 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:15.972405 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020773e-9b26-4979-90ab-00ded3ca3205 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:15.974008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0020773e-9b26-4979-90ab-00ded3ca3205 service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:15.985698 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1200/2388] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:24.088702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=71 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:25.981463 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:30.761882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695e711-6ed4-46f6-b19b-37f6f37367aa service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:30.764055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b695e711-6ed4-46f6-b19b-37f6f37367aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:30.778731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695e711-6ed4-46f6-b19b-37f6f37367aa service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:30.784158 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff09d2a-ed6f-41a1-a69b-9a30c752b544 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:30.784756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff09d2a-ed6f-41a1-a69b-9a30c752b544 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:30.785037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff09d2a-ed6f-41a1-a69b-9a30c752b544 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:30.785312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff09d2a-ed6f-41a1-a69b-9a30c752b544 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:30.786989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9ff09d2a-ed6f-41a1-a69b-9a30c752b544 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:30.848882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff09d2a-ed6f-41a1-a69b-9a30c752b544 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:30.850522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff09d2a-ed6f-41a1-a69b-9a30c752b544 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:30.859667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1189/2389] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:35.160210 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020773e-9b26-4979-90ab-00ded3ca3205 service nova] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:35.161547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0020773e-9b26-4979-90ab-00ded3ca3205 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:35.218158 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0020773e-9b26-4979-90ab-00ded3ca3205 service nova] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:35.221783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc809b-c840-41d0-9dc7-7dfd65b29cfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:35.222379 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc809b-c840-41d0-9dc7-7dfd65b29cfc service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:35.222660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc809b-c840-41d0-9dc7-7dfd65b29cfc service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:35.222931 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc809b-c840-41d0-9dc7-7dfd65b29cfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:35.224619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-3fbc809b-c840-41d0-9dc7-7dfd65b29cfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:35.282494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc809b-c840-41d0-9dc7-7dfd65b29cfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:35.283926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc809b-c840-41d0-9dc7-7dfd65b29cfc service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:35.292832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1201/2390] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:38.458156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff09d2a-ed6f-41a1-a69b-9a30c752b544 service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:38.470487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff09d2a-ed6f-41a1-a69b-9a30c752b544 service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:38.476115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e9501-06f7-4ab1-ab16-f541eecb8cb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:38.476905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e9501-06f7-4ab1-ab16-f541eecb8cb6 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:38.477252 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e9501-06f7-4ab1-ab16-f541eecb8cb6 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:38.477592 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453e9501-06f7-4ab1-ab16-f541eecb8cb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:38.479445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-453e9501-06f7-4ab1-ab16-f541eecb8cb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:38.492704 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e9501-06f7-4ab1-ab16-f541eecb8cb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:38.494479 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453e9501-06f7-4ab1-ab16-f541eecb8cb6 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:38.506206 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1190/2391] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:43.397412 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1470d9c-9444-47e2-9f19-df4db2a617c5 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:43.397943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1470d9c-9444-47e2-9f19-df4db2a617c5 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:43.398044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1470d9c-9444-47e2-9f19-df4db2a617c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:43.416819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b1470d9c-9444-47e2-9f19-df4db2a617c5 None None] MFA Rules not processed for user `96744eb6a9cf4895b6101e209d1fdd29`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:58:43.418038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b1470d9c-9444-47e2-9f19-df4db2a617c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:43.425263 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1202/2392] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:58:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:58:43.751094 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e9501-06f7-4ab1-ab16-f541eecb8cb6 service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:43.752619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-453e9501-06f7-4ab1-ab16-f541eecb8cb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:43.764517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453e9501-06f7-4ab1-ab16-f541eecb8cb6 service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:43.768198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f1c45-2320-4776-8574-3d221b5d6c63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:43.768874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f1c45-2320-4776-8574-3d221b5d6c63 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:43.769224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f1c45-2320-4776-8574-3d221b5d6c63 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:43.769565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f1c45-2320-4776-8574-3d221b5d6c63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:43.771420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ff3f1c45-2320-4776-8574-3d221b5d6c63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:43.781423 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f1c45-2320-4776-8574-3d221b5d6c63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:43.782875 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f1c45-2320-4776-8574-3d221b5d6c63 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:43.791097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1191/2393] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:53.411353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=51 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:53.787675 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=40 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:59.567092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1470d9c-9444-47e2-9f19-df4db2a617c5 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:58:59.585843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1470d9c-9444-47e2-9f19-df4db2a617c5 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:58:59.594642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f5471-c3c5-45a0-9ca2-1fa18dc3d33a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:58:59.595621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f5471-c3c5-45a0-9ca2-1fa18dc3d33a service nova] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:59.596108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f5471-c3c5-45a0-9ca2-1fa18dc3d33a service nova] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:59.596607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f5471-c3c5-45a0-9ca2-1fa18dc3d33a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:59.598667 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-833f5471-c3c5-45a0-9ca2-1fa18dc3d33a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:58:59.614380 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f5471-c3c5-45a0-9ca2-1fa18dc3d33a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:58:59.616874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833f5471-c3c5-45a0-9ca2-1fa18dc3d33a service nova] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:58:59.644364 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1203/2394] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:58:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:58:59.970696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d070f-956b-46b6-bef1-78c348a1ee5c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:58:59.970696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d070f-956b-46b6-bef1-78c348a1ee5c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:58:59.970696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d070f-956b-46b6-bef1-78c348a1ee5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:58:59.993075 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-af9d070f-956b-46b6-bef1-78c348a1ee5c None None] MFA Rules not processed for user `75642df28f5d4301b979f39482a9a11d`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:58:59.994375 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-af9d070f-956b-46b6-bef1-78c348a1ee5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:00.053977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1192/2395] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:58:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:00.804578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ae2dd-960a-40f2-a667-75b50c2bc1ea None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:00.805446 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ae2dd-960a-40f2-a667-75b50c2bc1ea None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:00.805727 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ae2dd-960a-40f2-a667-75b50c2bc1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:00.824630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-198ae2dd-960a-40f2-a667-75b50c2bc1ea None None] MFA Rules not processed for user `2e48a8c1b91142cf9d1af17f28e41961`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:00.825911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-198ae2dd-960a-40f2-a667-75b50c2bc1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:00.832843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1204/2396] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:10.046606 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=25 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:10.819473 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=9 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:15.668810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d070f-956b-46b6-bef1-78c348a1ee5c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:15.681375 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d070f-956b-46b6-bef1-78c348a1ee5c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:15.687518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d4908-f4d6-4de3-be92-9a33b87973c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:15.688129 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d4908-f4d6-4de3-be92-9a33b87973c1 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:15.688411 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d4908-f4d6-4de3-be92-9a33b87973c1 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:15.688685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d4908-f4d6-4de3-be92-9a33b87973c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:15.690427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-838d4908-f4d6-4de3-be92-9a33b87973c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:15.702899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d4908-f4d6-4de3-be92-9a33b87973c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:15.704646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d4908-f4d6-4de3-be92-9a33b87973c1 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:15.713799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1193/2397] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:16.809346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ae2dd-960a-40f2-a667-75b50c2bc1ea None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:16.810891 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-198ae2dd-960a-40f2-a667-75b50c2bc1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:16.822691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ae2dd-960a-40f2-a667-75b50c2bc1ea None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:16.826852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3410663-26f3-405c-9af8-d714f2462c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:16.827443 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3410663-26f3-405c-9af8-d714f2462c8c service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:16.827721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3410663-26f3-405c-9af8-d714f2462c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:16.827993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3410663-26f3-405c-9af8-d714f2462c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:16.829627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f3410663-26f3-405c-9af8-d714f2462c8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:16.891490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3410663-26f3-405c-9af8-d714f2462c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:16.892902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3410663-26f3-405c-9af8-d714f2462c8c service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:16.903763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1205/2398] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:21.027929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d4908-f4d6-4de3-be92-9a33b87973c1 service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:21.029325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-838d4908-f4d6-4de3-be92-9a33b87973c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:21.117112 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d4908-f4d6-4de3-be92-9a33b87973c1 service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:21.124600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec1467-653f-482d-92ca-af75e61df741 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:21.125077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3410663-26f3-405c-9af8-d714f2462c8c service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:21.125528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec1467-653f-482d-92ca-af75e61df741 service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:21.125528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec1467-653f-482d-92ca-af75e61df741 service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:21.125771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ec1467-653f-482d-92ca-af75e61df741 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:21.127740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-15ec1467-653f-482d-92ca-af75e61df741 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:21.129225 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f3410663-26f3-405c-9af8-d714f2462c8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:21.145326 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec1467-653f-482d-92ca-af75e61df741 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:21.147019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ec1467-653f-482d-92ca-af75e61df741 service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:21.147222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3410663-26f3-405c-9af8-d714f2462c8c service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:21.152498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185aa548-0e97-436e-9a82-8057a6782b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:21.153472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185aa548-0e97-436e-9a82-8057a6782b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:21.154298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185aa548-0e97-436e-9a82-8057a6782b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:21.154734 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185aa548-0e97-436e-9a82-8057a6782b58 admin admin] PATH_INFO: `/v3/users/9f07158e4fd1430e839a1592e1e11b58` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:21.161359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1194/2399] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:21.163096 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185aa548-0e97-436e-9a82-8057a6782b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f07158e4fd1430e839a1592e1e11b58)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:21.165816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185aa548-0e97-436e-9a82-8057a6782b58 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:21.239562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-185aa548-0e97-436e-9a82-8057a6782b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f07158e4fd1430e839a1592e1e11b58'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:21.247060 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-185aa548-0e97-436e-9a82-8057a6782b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f07158e4fd1430e839a1592e1e11b58'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:21.247725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-185aa548-0e97-436e-9a82-8057a6782b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f07158e4fd1430e839a1592e1e11b58'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:21.248665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-185aa548-0e97-436e-9a82-8057a6782b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f07158e4fd1430e839a1592e1e11b58'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:21.256692 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-185aa548-0e97-436e-9a82-8057a6782b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f07158e4fd1430e839a1592e1e11b58'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:21.277171 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1206/2400] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:21 2022] DELETE /identity/v3/users/9f07158e4fd1430e839a1592e1e11b58 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:21.473535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec1467-653f-482d-92ca-af75e61df741 service placement] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:21.486744 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ec1467-653f-482d-92ca-af75e61df741 service placement] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:21.551533 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c79dcd-f203-406f-81f3-3fa406dddad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:21.552380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c79dcd-f203-406f-81f3-3fa406dddad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:21.552743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c79dcd-f203-406f-81f3-3fa406dddad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:21.553138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c79dcd-f203-406f-81f3-3fa406dddad5 admin admin] PATH_INFO: `/v3/projects/127bce2080d54fe4871b5aaa6cc394b5` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:21.554456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c79dcd-f203-406f-81f3-3fa406dddad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=127bce2080d54fe4871b5aaa6cc394b5)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:21.556477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c79dcd-f203-406f-81f3-3fa406dddad5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:21.607146 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b0c79dcd-f203-406f-81f3-3fa406dddad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '127bce2080d54fe4871b5aaa6cc394b5', 'request_id': 'req-b0c79dcd-f203-406f-81f3-3fa406dddad5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:21.613805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b0c79dcd-f203-406f-81f3-3fa406dddad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '127bce2080d54fe4871b5aaa6cc394b5', 'request_id': 'req-b0c79dcd-f203-406f-81f3-3fa406dddad5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:21.619512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b0c79dcd-f203-406f-81f3-3fa406dddad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '127bce2080d54fe4871b5aaa6cc394b5', 'request_id': 'req-b0c79dcd-f203-406f-81f3-3fa406dddad5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:21.632698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b0c79dcd-f203-406f-81f3-3fa406dddad5 admin admin] The token cache is being invalidate because project 127bce2080d54fe4871b5aaa6cc394b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:21.632986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b0c79dcd-f203-406f-81f3-3fa406dddad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:21.633630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b0c79dcd-f203-406f-81f3-3fa406dddad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:21.635156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1195/2401] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:21 2022] DELETE /identity/v3/projects/127bce2080d54fe4871b5aaa6cc394b5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:21.640817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60f4c0-2978-4a07-a39b-df0f80253a61 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:21.641109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60f4c0-2978-4a07-a39b-df0f80253a61 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:21.641390 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60f4c0-2978-4a07-a39b-df0f80253a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:21.661927 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4e60f4c0-2978-4a07-a39b-df0f80253a61 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:21.663149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4e60f4c0-2978-4a07-a39b-df0f80253a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:21.733183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1207/2402] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:21.741376 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6d212-3c1b-475d-a6cb-cb7bac59fbea None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:21.741661 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6d212-3c1b-475d-a6cb-cb7bac59fbea None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:21.741969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6d212-3c1b-475d-a6cb-cb7bac59fbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:21.756805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:21.765707 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5de6d212-3c1b-475d-a6cb-cb7bac59fbea None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:21.769885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5de6d212-3c1b-475d-a6cb-cb7bac59fbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:21.780728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1196/2403] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:21.785014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60f4c0-2978-4a07-a39b-df0f80253a61 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:21.797680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60f4c0-2978-4a07-a39b-df0f80253a61 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:21.802792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd8abe-703d-4ccb-b207-9758fd4b9caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:21.803668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8abe-703d-4ccb-b207-9758fd4b9caf admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:21.804259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8abe-703d-4ccb-b207-9758fd4b9caf admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:21.804633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd8abe-703d-4ccb-b207-9758fd4b9caf admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:21.805396 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd8abe-703d-4ccb-b207-9758fd4b9caf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:21.805942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd8abe-703d-4ccb-b207-9758fd4b9caf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:21.807686 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd8abe-703d-4ccb-b207-9758fd4b9caf admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:21.810116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:21.824115 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1208/2404] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:59:21.829117 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6d212-3c1b-475d-a6cb-cb7bac59fbea None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:21.840233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6d212-3c1b-475d-a6cb-cb7bac59fbea None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:21.844883 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab15cc-1c33-4ced-99e4-e1f8748bcb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:21.845483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab15cc-1c33-4ced-99e4-e1f8748bcb67 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:21.845788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab15cc-1c33-4ced-99e4-e1f8748bcb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:21.846079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab15cc-1c33-4ced-99e4-e1f8748bcb67 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:21.846710 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab15cc-1c33-4ced-99e4-e1f8748bcb67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:21.848594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab15cc-1c33-4ced-99e4-e1f8748bcb67 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:21.875516 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1197/2405] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:21 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:21.880101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd8abe-703d-4ccb-b207-9758fd4b9caf admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:21.890968 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd8abe-703d-4ccb-b207-9758fd4b9caf admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:21.953888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc3169-6c65-4d31-a43f-c2edf23f4927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:21.954502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc3169-6c65-4d31-a43f-c2edf23f4927 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:21.954778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc3169-6c65-4d31-a43f-c2edf23f4927 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:21.955047 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc3169-6c65-4d31-a43f-c2edf23f4927 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:21.955673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc3169-6c65-4d31-a43f-c2edf23f4927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:21.957587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc3169-6c65-4d31-a43f-c2edf23f4927 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:21.978056 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1209/2406] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:21 2022] POST /identity/v3/users => generated 284 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:21.982123 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab15cc-1c33-4ced-99e4-e1f8748bcb67 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:21.992831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab15cc-1c33-4ced-99e4-e1f8748bcb67 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:21.999256 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5f6b3-34af-4f29-a42e-810be1168c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:21.999893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5f6b3-34af-4f29-a42e-810be1168c66 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:22.000141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5f6b3-34af-4f29-a42e-810be1168c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:22.000410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5f6b3-34af-4f29-a42e-810be1168c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:22.001139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5f6b3-34af-4f29-a42e-810be1168c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:22.002768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5f6b3-34af-4f29-a42e-810be1168c66 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:22.015497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1198/2407] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:22.019696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc3169-6c65-4d31-a43f-c2edf23f4927 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:22.029679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc3169-6c65-4d31-a43f-c2edf23f4927 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:22.034416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179496f-dffb-4c5e-8c91-de96d2ea185b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:22.035011 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179496f-dffb-4c5e-8c91-de96d2ea185b admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:22.035290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179496f-dffb-4c5e-8c91-de96d2ea185b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:22.035563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179496f-dffb-4c5e-8c91-de96d2ea185b admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:22.036205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179496f-dffb-4c5e-8c91-de96d2ea185b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:22.037845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1179496f-dffb-4c5e-8c91-de96d2ea185b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:22.048309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1210/2408] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:22.052364 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5f6b3-34af-4f29-a42e-810be1168c66 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:22.062748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5f6b3-34af-4f29-a42e-810be1168c66 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:22.067362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2109f90-e558-4943-a9e6-ffea4b327262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:22.068008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2109f90-e558-4943-a9e6-ffea4b327262 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:22.068288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2109f90-e558-4943-a9e6-ffea4b327262 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:22.068561 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2109f90-e558-4943-a9e6-ffea4b327262 admin admin] PATH_INFO: `/v3/projects/f74574e76a3f406c99649454ad1f4e3a/users/e1a929957a6d43499e06678ec124f632/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:22.083604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2109f90-e558-4943-a9e6-ffea4b327262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f74574e76a3f406c99649454ad1f4e3a, user_id=e1a929957a6d43499e06678ec124f632, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:22.086290 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2109f90-e558-4943-a9e6-ffea4b327262 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:22.099206 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1199/2409] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:22 2022] PUT /identity/v3/projects/f74574e76a3f406c99649454ad1f4e3a/users/e1a929957a6d43499e06678ec124f632/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:22.110822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd70dab-20cb-4ae0-9a15-014b7c9dc2cd None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:22.111184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd70dab-20cb-4ae0-9a15-014b7c9dc2cd None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:22.111531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd70dab-20cb-4ae0-9a15-014b7c9dc2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:22.142864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-2bd70dab-20cb-4ae0-9a15-014b7c9dc2cd None None] MFA Rules not processed for user `e1a929957a6d43499e06678ec124f632`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:22.144011 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2bd70dab-20cb-4ae0-9a15-014b7c9dc2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:22.207758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1211/2410] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:22.218883 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2109f90-e558-4943-a9e6-ffea4b327262 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:22.220552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e2109f90-e558-4943-a9e6-ffea4b327262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:22.276562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2109f90-e558-4943-a9e6-ffea4b327262 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:22.280242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bd4bda-54ff-4c52-bcd2-59a52a4b4347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:22.280839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd4bda-54ff-4c52-bcd2-59a52a4b4347 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:22.281121 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd4bda-54ff-4c52-bcd2-59a52a4b4347 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:22.281398 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bd4bda-54ff-4c52-bcd2-59a52a4b4347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:22.289600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd4bda-54ff-4c52-bcd2-59a52a4b4347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:22.291056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bd4bda-54ff-4c52-bcd2-59a52a4b4347 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:22.300270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1200/2411] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:30.750563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd70dab-20cb-4ae0-9a15-014b7c9dc2cd None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:30.811770 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd70dab-20cb-4ae0-9a15-014b7c9dc2cd None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:30.816271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:30.816897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:30.817199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:30.817499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] PATH_INFO: `/v3/users/e1a929957a6d43499e06678ec124f632` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:30.818710 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1a929957a6d43499e06678ec124f632)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:30.820620 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:30.822947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:30.867871 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1a929957a6d43499e06678ec124f632'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:30.875460 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1a929957a6d43499e06678ec124f632'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:30.876069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1a929957a6d43499e06678ec124f632'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:30.876668 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1a929957a6d43499e06678ec124f632'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:30.883585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1a929957a6d43499e06678ec124f632'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:30.901112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1212/2412] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:30 2022] DELETE /identity/v3/users/e1a929957a6d43499e06678ec124f632 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:30.906360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9566c-3579-4b25-87ef-ea447c1ac3d1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:30.906865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9566c-3579-4b25-87ef-ea447c1ac3d1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:30.906974 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9566c-3579-4b25-87ef-ea447c1ac3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:30.926507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-c1d9566c-3579-4b25-87ef-ea447c1ac3d1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:30.927687 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c1d9566c-3579-4b25-87ef-ea447c1ac3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:30.985542 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1201/2413] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:30.995175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:30.996672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:31.053549 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727853c-ba31-4846-af76-773c963b065a admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:31.057206 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd3560-7194-42e6-8562-088df0fb05c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:31.057845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd3560-7194-42e6-8562-088df0fb05c0 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:31.058128 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd3560-7194-42e6-8562-088df0fb05c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:31.058428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd3560-7194-42e6-8562-088df0fb05c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:31.067302 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd3560-7194-42e6-8562-088df0fb05c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:31.068736 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd3560-7194-42e6-8562-088df0fb05c0 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:31.078118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1213/2414] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:31.340188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9566c-3579-4b25-87ef-ea447c1ac3d1 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:31.342150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c1d9566c-3579-4b25-87ef-ea447c1ac3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:31.355900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9566c-3579-4b25-87ef-ea447c1ac3d1 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:31.367548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94184274-1247-43fb-990c-9e2709d5a0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:31.368224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94184274-1247-43fb-990c-9e2709d5a0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:31.368479 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94184274-1247-43fb-990c-9e2709d5a0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:31.368783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94184274-1247-43fb-990c-9e2709d5a0b0 admin admin] PATH_INFO: `/v3/projects/f74574e76a3f406c99649454ad1f4e3a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:31.370106 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94184274-1247-43fb-990c-9e2709d5a0b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f74574e76a3f406c99649454ad1f4e3a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:31.371989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94184274-1247-43fb-990c-9e2709d5a0b0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:31.415459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:31.423055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-94184274-1247-43fb-990c-9e2709d5a0b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f74574e76a3f406c99649454ad1f4e3a', 'request_id': 'req-94184274-1247-43fb-990c-9e2709d5a0b0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:31.429855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-94184274-1247-43fb-990c-9e2709d5a0b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f74574e76a3f406c99649454ad1f4e3a', 'request_id': 'req-94184274-1247-43fb-990c-9e2709d5a0b0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:31.435860 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-94184274-1247-43fb-990c-9e2709d5a0b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f74574e76a3f406c99649454ad1f4e3a', 'request_id': 'req-94184274-1247-43fb-990c-9e2709d5a0b0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:31.447843 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-94184274-1247-43fb-990c-9e2709d5a0b0 admin admin] The token cache is being invalidate because project f74574e76a3f406c99649454ad1f4e3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:31.448130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-94184274-1247-43fb-990c-9e2709d5a0b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:31.448745 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-94184274-1247-43fb-990c-9e2709d5a0b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:31.450800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1202/2415] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:31 2022] DELETE /identity/v3/projects/f74574e76a3f406c99649454ad1f4e3a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:31.455596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c30bf6-0dff-4a90-9ad5-9dc588fdbd8b None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:31.455912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c30bf6-0dff-4a90-9ad5-9dc588fdbd8b None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:31.456217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c30bf6-0dff-4a90-9ad5-9dc588fdbd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:31.478688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-55c30bf6-0dff-4a90-9ad5-9dc588fdbd8b None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:31.479926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-55c30bf6-0dff-4a90-9ad5-9dc588fdbd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:31.539821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1214/2416] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:31.548200 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453b901-91c1-40f5-9161-7f6874f599d8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:31.548514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453b901-91c1-40f5-9161-7f6874f599d8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:31.548790 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453b901-91c1-40f5-9161-7f6874f599d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:31.573500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2453b901-91c1-40f5-9161-7f6874f599d8 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:31.575418 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2453b901-91c1-40f5-9161-7f6874f599d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:31.583637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1203/2417] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:31.588012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c30bf6-0dff-4a90-9ad5-9dc588fdbd8b None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:31.597955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c30bf6-0dff-4a90-9ad5-9dc588fdbd8b None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:31.602696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525701dc-e872-4267-aff0-cbe44ef0cbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:31.603307 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525701dc-e872-4267-aff0-cbe44ef0cbf6 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:31.603590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525701dc-e872-4267-aff0-cbe44ef0cbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:31.603995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525701dc-e872-4267-aff0-cbe44ef0cbf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:31.604673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525701dc-e872-4267-aff0-cbe44ef0cbf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:31.604975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525701dc-e872-4267-aff0-cbe44ef0cbf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:31.606608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525701dc-e872-4267-aff0-cbe44ef0cbf6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:31.618177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1215/2418] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:31.622981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453b901-91c1-40f5-9161-7f6874f599d8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:31.631946 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453b901-91c1-40f5-9161-7f6874f599d8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:31.636500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131f007-d8e7-4a11-8c03-173895bd0dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:31.637657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131f007-d8e7-4a11-8c03-173895bd0dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:31.637983 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131f007-d8e7-4a11-8c03-173895bd0dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:31.638250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131f007-d8e7-4a11-8c03-173895bd0dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:31.638898 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131f007-d8e7-4a11-8c03-173895bd0dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:31.640773 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131f007-d8e7-4a11-8c03-173895bd0dd0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:31.668773 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1204/2419] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:31 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:59:31.672936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525701dc-e872-4267-aff0-cbe44ef0cbf6 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:31.682950 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525701dc-e872-4267-aff0-cbe44ef0cbf6 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:31.691131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:31.741722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe808887-f16d-48ae-b3d5-dd7afc219951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:31.742352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe808887-f16d-48ae-b3d5-dd7afc219951 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:31.742630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe808887-f16d-48ae-b3d5-dd7afc219951 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:31.742900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe808887-f16d-48ae-b3d5-dd7afc219951 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:31.743525 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe808887-f16d-48ae-b3d5-dd7afc219951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:31.745431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe808887-f16d-48ae-b3d5-dd7afc219951 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:31.767089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1216/2420] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:31 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:31.771170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131f007-d8e7-4a11-8c03-173895bd0dd0 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:31.781292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131f007-d8e7-4a11-8c03-173895bd0dd0 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:31.785882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3e310-a266-41f5-b783-f9f62fc08383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:31.786473 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3e310-a266-41f5-b783-f9f62fc08383 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:31.786754 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3e310-a266-41f5-b783-f9f62fc08383 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:31.787028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3e310-a266-41f5-b783-f9f62fc08383 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:31.787663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3e310-a266-41f5-b783-f9f62fc08383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:31.789521 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3e310-a266-41f5-b783-f9f62fc08383 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:31.800216 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1205/2421] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:59:31.804203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe808887-f16d-48ae-b3d5-dd7afc219951 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:31.814708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe808887-f16d-48ae-b3d5-dd7afc219951 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:31.819507 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f31d8e-5a25-4bbd-803f-cc68a5b712d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:31.820110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f31d8e-5a25-4bbd-803f-cc68a5b712d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:31.820419 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f31d8e-5a25-4bbd-803f-cc68a5b712d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:31.820694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f31d8e-5a25-4bbd-803f-cc68a5b712d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:31.821338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f31d8e-5a25-4bbd-803f-cc68a5b712d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:31.822957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f31d8e-5a25-4bbd-803f-cc68a5b712d8 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:31.834447 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1217/2422] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:31.838678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3e310-a266-41f5-b783-f9f62fc08383 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:31.848678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3e310-a266-41f5-b783-f9f62fc08383 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:31.853310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95472db-4a59-40ba-b18b-525a165ee908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:31.853978 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95472db-4a59-40ba-b18b-525a165ee908 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:31.854257 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95472db-4a59-40ba-b18b-525a165ee908 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:31.854529 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95472db-4a59-40ba-b18b-525a165ee908 admin admin] PATH_INFO: `/v3/projects/a824c0b07ddd407cba03c5e15544be58/users/1c3149feb4c740ad8a408bfbeb36c782/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:31.869220 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95472db-4a59-40ba-b18b-525a165ee908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a824c0b07ddd407cba03c5e15544be58, user_id=1c3149feb4c740ad8a408bfbeb36c782, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:31.872454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95472db-4a59-40ba-b18b-525a165ee908 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:31.886097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1206/2423] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:31 2022] PUT /identity/v3/projects/a824c0b07ddd407cba03c5e15544be58/users/1c3149feb4c740ad8a408bfbeb36c782/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:59:31.898210 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b724d36-1ffa-45b7-9f2c-a330638c3fc8 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:31.898497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b724d36-1ffa-45b7-9f2c-a330638c3fc8 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:31.898773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b724d36-1ffa-45b7-9f2c-a330638c3fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:31.926863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-6b724d36-1ffa-45b7-9f2c-a330638c3fc8 None None] MFA Rules not processed for user `1c3149feb4c740ad8a408bfbeb36c782`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:31.928061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-6b724d36-1ffa-45b7-9f2c-a330638c3fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:31.991552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1218/2424] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:31.996759 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95472db-4a59-40ba-b18b-525a165ee908 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:32.057219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95472db-4a59-40ba-b18b-525a165ee908 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:32.065078 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5d091-8c07-4199-a394-978f3ccf15b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:32.065685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5d091-8c07-4199-a394-978f3ccf15b9 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:32.066052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5d091-8c07-4199-a394-978f3ccf15b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:32.066345 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5d091-8c07-4199-a394-978f3ccf15b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:32.066973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5d091-8c07-4199-a394-978f3ccf15b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:32.068962 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5d091-8c07-4199-a394-978f3ccf15b9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:32.095069 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1207/2425] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:31 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:32.099190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b724d36-1ffa-45b7-9f2c-a330638c3fc8 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:32.110360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b724d36-1ffa-45b7-9f2c-a330638c3fc8 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:32.166085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48028022-3e1b-40a9-8b12-243ce7292038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:32.166689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48028022-3e1b-40a9-8b12-243ce7292038 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:32.166967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48028022-3e1b-40a9-8b12-243ce7292038 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:32.167238 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48028022-3e1b-40a9-8b12-243ce7292038 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:32.168366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48028022-3e1b-40a9-8b12-243ce7292038 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:32.170313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48028022-3e1b-40a9-8b12-243ce7292038 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:32.191226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1219/2426] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:32 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:59:32.195386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5d091-8c07-4199-a394-978f3ccf15b9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:32.209885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5d091-8c07-4199-a394-978f3ccf15b9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:32.210635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759f715-56bb-44e6-91b3-e2bedc23e917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:32.211224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759f715-56bb-44e6-91b3-e2bedc23e917 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:32.211498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759f715-56bb-44e6-91b3-e2bedc23e917 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:32.211808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759f715-56bb-44e6-91b3-e2bedc23e917 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:32.212429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759f715-56bb-44e6-91b3-e2bedc23e917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:32.214019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759f715-56bb-44e6-91b3-e2bedc23e917 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:32.225249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1208/2427] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:32.229438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48028022-3e1b-40a9-8b12-243ce7292038 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:32.239309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48028022-3e1b-40a9-8b12-243ce7292038 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:32.243989 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b0834-4d5b-4783-bdc6-1834493f3246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:32.244564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b0834-4d5b-4783-bdc6-1834493f3246 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:32.244843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b0834-4d5b-4783-bdc6-1834493f3246 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:32.245113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b0834-4d5b-4783-bdc6-1834493f3246 admin admin] PATH_INFO: `/v3/projects/7b4e3acbd3bc4aa788254e4b48ed3812/users/f3b6184965f346ddbcc213d2fd665628/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:32.248642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:32.261680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b0834-4d5b-4783-bdc6-1834493f3246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4e3acbd3bc4aa788254e4b48ed3812, user_id=f3b6184965f346ddbcc213d2fd665628, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:32.264852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b0834-4d5b-4783-bdc6-1834493f3246 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:32.276995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1220/2428] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:32 2022] PUT /identity/v3/projects/7b4e3acbd3bc4aa788254e4b48ed3812/users/f3b6184965f346ddbcc213d2fd665628/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:32.280904 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759f715-56bb-44e6-91b3-e2bedc23e917 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:32.290691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759f715-56bb-44e6-91b3-e2bedc23e917 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:32.328035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:32.347760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88316695-3bea-4fd2-aa2e-77a96788b3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:32.348369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88316695-3bea-4fd2-aa2e-77a96788b3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:32.348647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88316695-3bea-4fd2-aa2e-77a96788b3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:32.348919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88316695-3bea-4fd2-aa2e-77a96788b3b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:32.349553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88316695-3bea-4fd2-aa2e-77a96788b3b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:32.351176 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88316695-3bea-4fd2-aa2e-77a96788b3b9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:32.360765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1209/2429] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:32.364893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b0834-4d5b-4783-bdc6-1834493f3246 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:32.375555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b0834-4d5b-4783-bdc6-1834493f3246 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:32.380141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81743b-336a-4b79-a94e-dd11ca209da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:32.380780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81743b-336a-4b79-a94e-dd11ca209da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:32.381078 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81743b-336a-4b79-a94e-dd11ca209da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:32.381351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81743b-336a-4b79-a94e-dd11ca209da2 admin admin] PATH_INFO: `/v3/projects/7b4e3acbd3bc4aa788254e4b48ed3812/users/f3b6184965f346ddbcc213d2fd665628/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:32.383360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81743b-336a-4b79-a94e-dd11ca209da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4e3acbd3bc4aa788254e4b48ed3812, user_id=f3b6184965f346ddbcc213d2fd665628, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:32.386048 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81743b-336a-4b79-a94e-dd11ca209da2 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:32.396714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1221/2430] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:32 2022] PUT /identity/v3/projects/7b4e3acbd3bc4aa788254e4b48ed3812/users/f3b6184965f346ddbcc213d2fd665628/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:32.408678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bada343-f62d-4d22-9004-1a30008ca8e8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:32.408967 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bada343-f62d-4d22-9004-1a30008ca8e8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:32.409262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bada343-f62d-4d22-9004-1a30008ca8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:32.436752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-3bada343-f62d-4d22-9004-1a30008ca8e8 None None] MFA Rules not processed for user `f3b6184965f346ddbcc213d2fd665628`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:32.437983 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3bada343-f62d-4d22-9004-1a30008ca8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:32.504184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1210/2431] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:32.515576 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81743b-336a-4b79-a94e-dd11ca209da2 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:32.517307 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-da81743b-336a-4b79-a94e-dd11ca209da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:32.576166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81743b-336a-4b79-a94e-dd11ca209da2 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:32.579863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd43b50-d1ad-4646-8e2c-b3b04fc6fa13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:32.580518 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd43b50-d1ad-4646-8e2c-b3b04fc6fa13 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:32.580799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd43b50-d1ad-4646-8e2c-b3b04fc6fa13 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:32.581064 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd43b50-d1ad-4646-8e2c-b3b04fc6fa13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:32.590267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd43b50-d1ad-4646-8e2c-b3b04fc6fa13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:32.591676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd43b50-d1ad-4646-8e2c-b3b04fc6fa13 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:32.600917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1222/2432] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:33.008601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bada343-f62d-4d22-9004-1a30008ca8e8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:33.072514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bada343-f62d-4d22-9004-1a30008ca8e8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:33.077097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:33.077727 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:33.078030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:33.078301 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] PATH_INFO: `/v3/users/1c3149feb4c740ad8a408bfbeb36c782` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:33.079410 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c3149feb4c740ad8a408bfbeb36c782)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:33.081324 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:33.126871 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c3149feb4c740ad8a408bfbeb36c782'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.134272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c3149feb4c740ad8a408bfbeb36c782'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.134902 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c3149feb4c740ad8a408bfbeb36c782'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.135619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c3149feb4c740ad8a408bfbeb36c782'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.142669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c3149feb4c740ad8a408bfbeb36c782'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.143778 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:33.162611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1211/2433] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:33 2022] DELETE /identity/v3/users/1c3149feb4c740ad8a408bfbeb36c782 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:33.167161 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b047147-a993-4587-917a-7d646387eb77 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:33.167677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b047147-a993-4587-917a-7d646387eb77 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:33.167816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b047147-a993-4587-917a-7d646387eb77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:33.186531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4b047147-a993-4587-917a-7d646387eb77 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:33.187698 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4b047147-a993-4587-917a-7d646387eb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:33.245413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1223/2434] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:33.255402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:33.256873 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:33.312742 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b70ef-d8fc-4873-be9b-ca63acd6f037 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:33.316291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e715be-cf95-4b6c-9ba5-9b855e9f02f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:33.316887 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e715be-cf95-4b6c-9ba5-9b855e9f02f3 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:33.317168 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e715be-cf95-4b6c-9ba5-9b855e9f02f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:33.317440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e715be-cf95-4b6c-9ba5-9b855e9f02f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:33.326257 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e715be-cf95-4b6c-9ba5-9b855e9f02f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:33.327676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e715be-cf95-4b6c-9ba5-9b855e9f02f3 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:33.336879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1212/2435] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:59:33.614140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b047147-a993-4587-917a-7d646387eb77 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:33.615810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4b047147-a993-4587-917a-7d646387eb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:33.628056 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b047147-a993-4587-917a-7d646387eb77 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:33.632678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:33.633325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:33.633586 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:33.633895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] PATH_INFO: `/v3/projects/a824c0b07ddd407cba03c5e15544be58` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:33.635038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a824c0b07ddd407cba03c5e15544be58)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:33.637768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:33.685086 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a824c0b07ddd407cba03c5e15544be58', 'request_id': 'req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.691446 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a824c0b07ddd407cba03c5e15544be58', 'request_id': 'req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.700534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a824c0b07ddd407cba03c5e15544be58', 'request_id': 'req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.707160 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:33.712688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] The token cache is being invalidate because project a824c0b07ddd407cba03c5e15544be58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:33.713189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.714294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.716778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1224/2436] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:33 2022] DELETE /identity/v3/projects/a824c0b07ddd407cba03c5e15544be58 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:33.720766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e715be-cf95-4b6c-9ba5-9b855e9f02f3 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:33.730427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e715be-cf95-4b6c-9ba5-9b855e9f02f3 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:33.795775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ef0c4-ad68-42ed-9876-1ed2fb42b70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:33.796438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ef0c4-ad68-42ed-9876-1ed2fb42b70a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:33.796718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ef0c4-ad68-42ed-9876-1ed2fb42b70a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:33.796992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ef0c4-ad68-42ed-9876-1ed2fb42b70a admin admin] PATH_INFO: `/v3/users/f3b6184965f346ddbcc213d2fd665628` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:33.798157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ef0c4-ad68-42ed-9876-1ed2fb42b70a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b6184965f346ddbcc213d2fd665628)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:33.800272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ef0c4-ad68-42ed-9876-1ed2fb42b70a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:33.847669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4f6ef0c4-ad68-42ed-9876-1ed2fb42b70a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b6184965f346ddbcc213d2fd665628'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.858014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4f6ef0c4-ad68-42ed-9876-1ed2fb42b70a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b6184965f346ddbcc213d2fd665628'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.858549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4f6ef0c4-ad68-42ed-9876-1ed2fb42b70a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b6184965f346ddbcc213d2fd665628'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.859262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4f6ef0c4-ad68-42ed-9876-1ed2fb42b70a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b6184965f346ddbcc213d2fd665628'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.867136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4f6ef0c4-ad68-42ed-9876-1ed2fb42b70a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b6184965f346ddbcc213d2fd665628'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:33.885090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1213/2437] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:33 2022] DELETE /identity/v3/users/f3b6184965f346ddbcc213d2fd665628 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:34.159825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:34.161370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:34.176503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc0cd7-6fe3-42eb-8915-b0ee0beee69e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:34.235266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ef659-4db6-4e47-a177-7f7fc086ca33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:34.235889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ef659-4db6-4e47-a177-7f7fc086ca33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.236164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ef659-4db6-4e47-a177-7f7fc086ca33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.236433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ef659-4db6-4e47-a177-7f7fc086ca33 admin admin] PATH_INFO: `/v3/projects/7b4e3acbd3bc4aa788254e4b48ed3812` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.237531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ef659-4db6-4e47-a177-7f7fc086ca33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b4e3acbd3bc4aa788254e4b48ed3812)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:34.239461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ef659-4db6-4e47-a177-7f7fc086ca33 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:34.287350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5f7ef659-4db6-4e47-a177-7f7fc086ca33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4e3acbd3bc4aa788254e4b48ed3812', 'request_id': 'req-5f7ef659-4db6-4e47-a177-7f7fc086ca33'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:34.293719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5f7ef659-4db6-4e47-a177-7f7fc086ca33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4e3acbd3bc4aa788254e4b48ed3812', 'request_id': 'req-5f7ef659-4db6-4e47-a177-7f7fc086ca33'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:34.301632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5f7ef659-4db6-4e47-a177-7f7fc086ca33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4e3acbd3bc4aa788254e4b48ed3812', 'request_id': 'req-5f7ef659-4db6-4e47-a177-7f7fc086ca33'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:34.311938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5f7ef659-4db6-4e47-a177-7f7fc086ca33 admin admin] The token cache is being invalidate because project 7b4e3acbd3bc4aa788254e4b48ed3812 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:34.312227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5f7ef659-4db6-4e47-a177-7f7fc086ca33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:34.312919 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5f7ef659-4db6-4e47-a177-7f7fc086ca33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:34.314441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1225/2438] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:34 2022] DELETE /identity/v3/projects/7b4e3acbd3bc4aa788254e4b48ed3812 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:34.319235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cb42f-3a0e-4351-b8a2-61ff5edc585f None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.319711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cb42f-3a0e-4351-b8a2-61ff5edc585f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.319818 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cb42f-3a0e-4351-b8a2-61ff5edc585f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.327702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:34.342383 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7b2cb42f-3a0e-4351-b8a2-61ff5edc585f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:34.343610 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7b2cb42f-3a0e-4351-b8a2-61ff5edc585f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:34.400194 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1214/2439] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:34.408603 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867334d4-ca6d-4135-9f4f-4e4e641ee1cc None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.408825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867334d4-ca6d-4135-9f4f-4e4e641ee1cc None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.409101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867334d4-ca6d-4135-9f4f-4e4e641ee1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.429346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-867334d4-ca6d-4135-9f4f-4e4e641ee1cc None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:34.430570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-867334d4-ca6d-4135-9f4f-4e4e641ee1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:34.438515 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1226/2440] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:34.442748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cb42f-3a0e-4351-b8a2-61ff5edc585f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:34.453205 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cb42f-3a0e-4351-b8a2-61ff5edc585f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:34.457713 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066cdec-2a54-4df3-ac6f-063c697f3af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:34.458336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066cdec-2a54-4df3-ac6f-063c697f3af4 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.458618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066cdec-2a54-4df3-ac6f-063c697f3af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.458889 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066cdec-2a54-4df3-ac6f-063c697f3af4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.459535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066cdec-2a54-4df3-ac6f-063c697f3af4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:34.459837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066cdec-2a54-4df3-ac6f-063c697f3af4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:34.461428 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066cdec-2a54-4df3-ac6f-063c697f3af4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:34.473858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1215/2441] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:34.483199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867334d4-ca6d-4135-9f4f-4e4e641ee1cc None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:34.492549 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867334d4-ca6d-4135-9f4f-4e4e641ee1cc None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:34.497759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214f31c-07e9-425b-aac6-e3fce491230c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:34.498394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214f31c-07e9-425b-aac6-e3fce491230c admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.498671 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214f31c-07e9-425b-aac6-e3fce491230c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.498939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214f31c-07e9-425b-aac6-e3fce491230c admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.499579 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214f31c-07e9-425b-aac6-e3fce491230c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:34.501461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214f31c-07e9-425b-aac6-e3fce491230c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:34.528420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1227/2442] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:34 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:59:34.532644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066cdec-2a54-4df3-ac6f-063c697f3af4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:34.543562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066cdec-2a54-4df3-ac6f-063c697f3af4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:34.600565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a17829-9877-4a35-92c3-bd8d64d069fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:34.601168 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a17829-9877-4a35-92c3-bd8d64d069fd admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.601447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a17829-9877-4a35-92c3-bd8d64d069fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.601720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a17829-9877-4a35-92c3-bd8d64d069fd admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.602374 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a17829-9877-4a35-92c3-bd8d64d069fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:34.604297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a17829-9877-4a35-92c3-bd8d64d069fd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:34.624914 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1216/2443] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:34 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:34.629609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214f31c-07e9-425b-aac6-e3fce491230c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:34.639879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214f31c-07e9-425b-aac6-e3fce491230c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:34.644499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00dcf9-2f3d-4b08-a6ee-23c666727075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:34.645089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00dcf9-2f3d-4b08-a6ee-23c666727075 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.645365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00dcf9-2f3d-4b08-a6ee-23c666727075 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.645634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00dcf9-2f3d-4b08-a6ee-23c666727075 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.646318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc00dcf9-2f3d-4b08-a6ee-23c666727075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:34.647904 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc00dcf9-2f3d-4b08-a6ee-23c666727075 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:34.658330 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1228/2444] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:59:34.662429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a17829-9877-4a35-92c3-bd8d64d069fd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:34.673486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a17829-9877-4a35-92c3-bd8d64d069fd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:34.678138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec55c9a-0d9e-407f-9125-0ed9e41584d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:34.678733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec55c9a-0d9e-407f-9125-0ed9e41584d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.679010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec55c9a-0d9e-407f-9125-0ed9e41584d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.679277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec55c9a-0d9e-407f-9125-0ed9e41584d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.679909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec55c9a-0d9e-407f-9125-0ed9e41584d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:34.681526 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec55c9a-0d9e-407f-9125-0ed9e41584d4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:34.693135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1217/2445] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:34.697298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00dcf9-2f3d-4b08-a6ee-23c666727075 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:34.707283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc00dcf9-2f3d-4b08-a6ee-23c666727075 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:34.711912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5c3a0-f892-4bf7-84c4-9f073a7c74be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:34.712554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5c3a0-f892-4bf7-84c4-9f073a7c74be admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.712834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5c3a0-f892-4bf7-84c4-9f073a7c74be admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.713109 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5c3a0-f892-4bf7-84c4-9f073a7c74be admin admin] PATH_INFO: `/v3/projects/1fdd895567ff478c8c776d74576be890/users/c497e4e659234ff789fe7eb7181a5d7f/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.729190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5c3a0-f892-4bf7-84c4-9f073a7c74be admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fdd895567ff478c8c776d74576be890, user_id=c497e4e659234ff789fe7eb7181a5d7f, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:34.732415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5c3a0-f892-4bf7-84c4-9f073a7c74be admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:34.745989 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1229/2446] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:34 2022] PUT /identity/v3/projects/1fdd895567ff478c8c776d74576be890/users/c497e4e659234ff789fe7eb7181a5d7f/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:34.751041 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b312b-6641-4ede-b423-55a8678dc22e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.751323 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b312b-6641-4ede-b423-55a8678dc22e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.751600 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b312b-6641-4ede-b423-55a8678dc22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.770636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-506b312b-6641-4ede-b423-55a8678dc22e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:34.771826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-506b312b-6641-4ede-b423-55a8678dc22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:34.836723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1218/2447] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:34.847437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5c3a0-f892-4bf7-84c4-9f073a7c74be admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:34.849177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-88b5c3a0-f892-4bf7-84c4-9f073a7c74be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:34.877250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:34.908587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5c3a0-f892-4bf7-84c4-9f073a7c74be admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:34.912317 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79080d61-b6d2-4a8d-9d46-40b59f4d48ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:34.912987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79080d61-b6d2-4a8d-9d46-40b59f4d48ac service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:34.913202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79080d61-b6d2-4a8d-9d46-40b59f4d48ac service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:34.913471 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79080d61-b6d2-4a8d-9d46-40b59f4d48ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:34.922603 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79080d61-b6d2-4a8d-9d46-40b59f4d48ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:34.924008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79080d61-b6d2-4a8d-9d46-40b59f4d48ac service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:34.933232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1230/2448] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:35.571915 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b002f-51ac-45e0-b10f-09b16b2fc71b None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.572455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b002f-51ac-45e0-b10f-09b16b2fc71b None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.572455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b002f-51ac-45e0-b10f-09b16b2fc71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.593979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:35.603787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-912b002f-51ac-45e0-b10f-09b16b2fc71b None None] MFA Rules not processed for user `c497e4e659234ff789fe7eb7181a5d7f`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:35.606390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-912b002f-51ac-45e0-b10f-09b16b2fc71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:35.675635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1219/2449] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:35.680186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79080d61-b6d2-4a8d-9d46-40b59f4d48ac service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.690415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79080d61-b6d2-4a8d-9d46-40b59f4d48ac service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.695676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b3418-2b42-4f0a-b534-c1135dba15a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.696276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b3418-2b42-4f0a-b534-c1135dba15a5 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.696551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b3418-2b42-4f0a-b534-c1135dba15a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.696847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b3418-2b42-4f0a-b534-c1135dba15a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.697469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b3418-2b42-4f0a-b534-c1135dba15a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.699444 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b3418-2b42-4f0a-b534-c1135dba15a5 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.730873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1231/2450] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:35.732202 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b002f-51ac-45e0-b10f-09b16b2fc71b None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.743076 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b002f-51ac-45e0-b10f-09b16b2fc71b None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.799893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17982ac5-0d7f-46c0-92b4-f3f014954020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.800504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17982ac5-0d7f-46c0-92b4-f3f014954020 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.800786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17982ac5-0d7f-46c0-92b4-f3f014954020 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.801063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17982ac5-0d7f-46c0-92b4-f3f014954020 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.801692 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17982ac5-0d7f-46c0-92b4-f3f014954020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.803769 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17982ac5-0d7f-46c0-92b4-f3f014954020 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.826007 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1220/2451] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:35 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:35.830092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b3418-2b42-4f0a-b534-c1135dba15a5 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.840401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b3418-2b42-4f0a-b534-c1135dba15a5 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.844837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e652f0a-2341-4e47-8d04-e3237d3c6476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.845423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e652f0a-2341-4e47-8d04-e3237d3c6476 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.845698 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e652f0a-2341-4e47-8d04-e3237d3c6476 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.846007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e652f0a-2341-4e47-8d04-e3237d3c6476 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.846665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e652f0a-2341-4e47-8d04-e3237d3c6476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.848247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e652f0a-2341-4e47-8d04-e3237d3c6476 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.859311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1232/2452] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:35.863292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17982ac5-0d7f-46c0-92b4-f3f014954020 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.873924 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17982ac5-0d7f-46c0-92b4-f3f014954020 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.878506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45bd65-537b-4343-b13b-f4048ca2d947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.879148 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45bd65-537b-4343-b13b-f4048ca2d947 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.879432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45bd65-537b-4343-b13b-f4048ca2d947 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.879708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45bd65-537b-4343-b13b-f4048ca2d947 admin admin] PATH_INFO: `/v3/projects/4e58747579d04e3d8b2af24ad3510e89/users/f1b3b8973f3943cabb176f07b46b1e03/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.894281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45bd65-537b-4343-b13b-f4048ca2d947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e58747579d04e3d8b2af24ad3510e89, user_id=f1b3b8973f3943cabb176f07b46b1e03, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.896982 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45bd65-537b-4343-b13b-f4048ca2d947 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.912580 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1221/2453] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:35 2022] PUT /identity/v3/projects/4e58747579d04e3d8b2af24ad3510e89/users/f1b3b8973f3943cabb176f07b46b1e03/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:35.916520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e652f0a-2341-4e47-8d04-e3237d3c6476 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:35.927530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e652f0a-2341-4e47-8d04-e3237d3c6476 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:35.984773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00f8fd0-7276-4cb7-9554-21d99f2e8d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:35.985301 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f8fd0-7276-4cb7-9554-21d99f2e8d3d admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:35.985577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f8fd0-7276-4cb7-9554-21d99f2e8d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:35.985886 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00f8fd0-7276-4cb7-9554-21d99f2e8d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:35.986530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f8fd0-7276-4cb7-9554-21d99f2e8d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:35.988156 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00f8fd0-7276-4cb7-9554-21d99f2e8d3d admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:35.997943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1233/2454] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:36.002288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45bd65-537b-4343-b13b-f4048ca2d947 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.012643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45bd65-537b-4343-b13b-f4048ca2d947 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.017303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215dcd2-f1cd-434a-aa89-4c5b0129db6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.018012 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215dcd2-f1cd-434a-aa89-4c5b0129db6c admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.018295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215dcd2-f1cd-434a-aa89-4c5b0129db6c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.018568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6215dcd2-f1cd-434a-aa89-4c5b0129db6c admin admin] PATH_INFO: `/v3/projects/4e58747579d04e3d8b2af24ad3510e89/users/f1b3b8973f3943cabb176f07b46b1e03/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.020478 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215dcd2-f1cd-434a-aa89-4c5b0129db6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e58747579d04e3d8b2af24ad3510e89, user_id=f1b3b8973f3943cabb176f07b46b1e03, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.023151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6215dcd2-f1cd-434a-aa89-4c5b0129db6c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.033685 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1222/2455] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:36 2022] PUT /identity/v3/projects/4e58747579d04e3d8b2af24ad3510e89/users/f1b3b8973f3943cabb176f07b46b1e03/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:36.651995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69ed8c-bd23-4929-9022-0c71e76fae5f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.652830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69ed8c-bd23-4929-9022-0c71e76fae5f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.653108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69ed8c-bd23-4929-9022-0c71e76fae5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.681949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-7b69ed8c-bd23-4929-9022-0c71e76fae5f None None] MFA Rules not processed for user `f1b3b8973f3943cabb176f07b46b1e03`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:36.683217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7b69ed8c-bd23-4929-9022-0c71e76fae5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:36.757828 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1234/2456] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:36.770404 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215dcd2-f1cd-434a-aa89-4c5b0129db6c admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:36.772105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6215dcd2-f1cd-434a-aa89-4c5b0129db6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:36.834987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6215dcd2-f1cd-434a-aa89-4c5b0129db6c admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:36.838751 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fb7b9-44c4-494e-b6ba-11731c6516eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:36.839360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fb7b9-44c4-494e-b6ba-11731c6516eb service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:36.839941 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fb7b9-44c4-494e-b6ba-11731c6516eb service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:36.840216 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fb7b9-44c4-494e-b6ba-11731c6516eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:36.844297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:36.896838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fb7b9-44c4-494e-b6ba-11731c6516eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:36.898305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fb7b9-44c4-494e-b6ba-11731c6516eb service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:36.907434 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1223/2457] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:38.268982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69ed8c-bd23-4929-9022-0c71e76fae5f None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:38.270384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7b69ed8c-bd23-4929-9022-0c71e76fae5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:38.282320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69ed8c-bd23-4929-9022-0c71e76fae5f None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:38.285875 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed6d14-ec66-4ec7-8588-aa742671dc05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:38.286472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed6d14-ec66-4ec7-8588-aa742671dc05 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:38.286752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed6d14-ec66-4ec7-8588-aa742671dc05 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:38.287028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed6d14-ec66-4ec7-8588-aa742671dc05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:38.296075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed6d14-ec66-4ec7-8588-aa742671dc05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:38.297488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed6d14-ec66-4ec7-8588-aa742671dc05 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:38.306853 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1235/2458] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:38 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:46.903212 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=25 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:48.302794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:51.303806 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fb7b9-44c4-494e-b6ba-11731c6516eb service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:51.365487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fb7b9-44c4-494e-b6ba-11731c6516eb service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:51.369960 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:51.371136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:51.371413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:51.371681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] PATH_INFO: `/v3/users/c497e4e659234ff789fe7eb7181a5d7f` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:51.372777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c497e4e659234ff789fe7eb7181a5d7f)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:51.374708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:51.420069 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c497e4e659234ff789fe7eb7181a5d7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.427219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c497e4e659234ff789fe7eb7181a5d7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.427821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c497e4e659234ff789fe7eb7181a5d7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.428449 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c497e4e659234ff789fe7eb7181a5d7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.435240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c497e4e659234ff789fe7eb7181a5d7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.452337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1224/2459] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:51 2022] DELETE /identity/v3/users/c497e4e659234ff789fe7eb7181a5d7f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:51.623123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed6d14-ec66-4ec7-8588-aa742671dc05 service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:51.635545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed6d14-ec66-4ec7-8588-aa742671dc05 service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:51.692839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:51.693465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:51.693799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:51.694079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] PATH_INFO: `/v3/projects/1fdd895567ff478c8c776d74576be890` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:51.695202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fdd895567ff478c8c776d74576be890)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:51.697093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:51.743818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fdd895567ff478c8c776d74576be890', 'request_id': 'req-47cf9f47-b30b-482d-9495-acfaec78a48b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.750584 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fdd895567ff478c8c776d74576be890', 'request_id': 'req-47cf9f47-b30b-482d-9495-acfaec78a48b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.757697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fdd895567ff478c8c776d74576be890', 'request_id': 'req-47cf9f47-b30b-482d-9495-acfaec78a48b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.768606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] The token cache is being invalidate because project 1fdd895567ff478c8c776d74576be890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:51.768896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.769551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.771077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1236/2460] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:51 2022] DELETE /identity/v3/projects/1fdd895567ff478c8c776d74576be890 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:51.775078 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:51.776485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:51.790162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da806ae-e056-4c3e-b031-7facba6611fd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:51.849698 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43ce8d-ea13-4037-bbb7-6cfe8ba7c7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:51.850359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43ce8d-ea13-4037-bbb7-6cfe8ba7c7b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:51.850641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43ce8d-ea13-4037-bbb7-6cfe8ba7c7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:51.850910 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43ce8d-ea13-4037-bbb7-6cfe8ba7c7b5 admin admin] PATH_INFO: `/v3/users/f1b3b8973f3943cabb176f07b46b1e03` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:51.852034 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43ce8d-ea13-4037-bbb7-6cfe8ba7c7b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b3b8973f3943cabb176f07b46b1e03)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:51.854120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43ce8d-ea13-4037-bbb7-6cfe8ba7c7b5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:51.875546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:51.901002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ab43ce8d-ea13-4037-bbb7-6cfe8ba7c7b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b3b8973f3943cabb176f07b46b1e03'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.909252 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ab43ce8d-ea13-4037-bbb7-6cfe8ba7c7b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b3b8973f3943cabb176f07b46b1e03'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.910114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ab43ce8d-ea13-4037-bbb7-6cfe8ba7c7b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b3b8973f3943cabb176f07b46b1e03'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.910758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ab43ce8d-ea13-4037-bbb7-6cfe8ba7c7b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b3b8973f3943cabb176f07b46b1e03'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.918731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-ab43ce8d-ea13-4037-bbb7-6cfe8ba7c7b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b3b8973f3943cabb176f07b46b1e03'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:51.936356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1225/2461] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 13:59:51 2022] DELETE /identity/v3/users/f1b3b8973f3943cabb176f07b46b1e03 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:52.110613 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.112162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:52.133503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf9f47-b30b-482d-9495-acfaec78a48b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.192221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994afad2-fc21-4fbd-aca7-7ff91479824c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.192874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994afad2-fc21-4fbd-aca7-7ff91479824c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.193149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994afad2-fc21-4fbd-aca7-7ff91479824c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.193423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994afad2-fc21-4fbd-aca7-7ff91479824c admin admin] PATH_INFO: `/v3/projects/4e58747579d04e3d8b2af24ad3510e89` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.194942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994afad2-fc21-4fbd-aca7-7ff91479824c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e58747579d04e3d8b2af24ad3510e89)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.196885 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994afad2-fc21-4fbd-aca7-7ff91479824c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.233917 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:52.248235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-994afad2-fc21-4fbd-aca7-7ff91479824c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e58747579d04e3d8b2af24ad3510e89', 'request_id': 'req-994afad2-fc21-4fbd-aca7-7ff91479824c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:52.255130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-994afad2-fc21-4fbd-aca7-7ff91479824c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e58747579d04e3d8b2af24ad3510e89', 'request_id': 'req-994afad2-fc21-4fbd-aca7-7ff91479824c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:52.263335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-994afad2-fc21-4fbd-aca7-7ff91479824c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e58747579d04e3d8b2af24ad3510e89', 'request_id': 'req-994afad2-fc21-4fbd-aca7-7ff91479824c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:52.274768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-994afad2-fc21-4fbd-aca7-7ff91479824c admin admin] The token cache is being invalidate because project 4e58747579d04e3d8b2af24ad3510e89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:59:52.275102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-994afad2-fc21-4fbd-aca7-7ff91479824c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:52.275778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-994afad2-fc21-4fbd-aca7-7ff91479824c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:59:52.277421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1237/2462] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 13:59:52 2022] DELETE /identity/v3/projects/4e58747579d04e3d8b2af24ad3510e89 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:52.282733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1ec50-3756-4d2c-9302-2f5cedc2ea0a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.283025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1ec50-3756-4d2c-9302-2f5cedc2ea0a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.283347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1ec50-3756-4d2c-9302-2f5cedc2ea0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.303353 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5ce1ec50-3756-4d2c-9302-2f5cedc2ea0a None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:52.304603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5ce1ec50-3756-4d2c-9302-2f5cedc2ea0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:52.362785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1226/2463] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:52.372115 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b96087-48a4-4e84-88e8-3a64bf1d2b9a None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.372397 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b96087-48a4-4e84-88e8-3a64bf1d2b9a None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.372666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b96087-48a4-4e84-88e8-3a64bf1d2b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.391984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-56b96087-48a4-4e84-88e8-3a64bf1d2b9a None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:52.393168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-56b96087-48a4-4e84-88e8-3a64bf1d2b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:52.400948 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1238/2464] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:52.405082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1ec50-3756-4d2c-9302-2f5cedc2ea0a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.414787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1ec50-3756-4d2c-9302-2f5cedc2ea0a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.419884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb32ed-796f-411e-89e6-cf55da11682e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.420483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb32ed-796f-411e-89e6-cf55da11682e admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.420763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb32ed-796f-411e-89e6-cf55da11682e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.421037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb32ed-796f-411e-89e6-cf55da11682e admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.421686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb32ed-796f-411e-89e6-cf55da11682e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:59:52.422023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb32ed-796f-411e-89e6-cf55da11682e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.423624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb32ed-796f-411e-89e6-cf55da11682e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.435207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1227/2465] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 13:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:59:52.439164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b96087-48a4-4e84-88e8-3a64bf1d2b9a None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.448149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b96087-48a4-4e84-88e8-3a64bf1d2b9a None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.453333 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f693338-ca9d-4333-8cda-4243f1d60b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.453979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f693338-ca9d-4333-8cda-4243f1d60b4f admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.454264 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f693338-ca9d-4333-8cda-4243f1d60b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.454540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f693338-ca9d-4333-8cda-4243f1d60b4f admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.455169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f693338-ca9d-4333-8cda-4243f1d60b4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.457088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f693338-ca9d-4333-8cda-4243f1d60b4f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.484777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1239/2466] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:52 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:52.488920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb32ed-796f-411e-89e6-cf55da11682e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.499292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb32ed-796f-411e-89e6-cf55da11682e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.555421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7a87c-e196-4ef1-8dfe-b3d18df15eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.555953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7a87c-e196-4ef1-8dfe-b3d18df15eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.556233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7a87c-e196-4ef1-8dfe-b3d18df15eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.556507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7a87c-e196-4ef1-8dfe-b3d18df15eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.557139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7a87c-e196-4ef1-8dfe-b3d18df15eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.559576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d7a87c-e196-4ef1-8dfe-b3d18df15eb8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.582048 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1228/2467] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:52 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:52.586207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f693338-ca9d-4333-8cda-4243f1d60b4f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.596580 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f693338-ca9d-4333-8cda-4243f1d60b4f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.601214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b73297-1050-4255-bc1d-85845cff79aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.601846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b73297-1050-4255-bc1d-85845cff79aa admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.602127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b73297-1050-4255-bc1d-85845cff79aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.602401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b73297-1050-4255-bc1d-85845cff79aa admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.603042 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b73297-1050-4255-bc1d-85845cff79aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.604670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b73297-1050-4255-bc1d-85845cff79aa admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.614393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1240/2468] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:52.618150 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7a87c-e196-4ef1-8dfe-b3d18df15eb8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.628543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7a87c-e196-4ef1-8dfe-b3d18df15eb8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.633181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f7ca3-d671-4661-902c-ffc1ed707c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.633796 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f7ca3-d671-4661-902c-ffc1ed707c9e admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.634078 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f7ca3-d671-4661-902c-ffc1ed707c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.634348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f7ca3-d671-4661-902c-ffc1ed707c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.634989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f7ca3-d671-4661-902c-ffc1ed707c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.636632 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f7ca3-d671-4661-902c-ffc1ed707c9e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.647011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1229/2469] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:52.651180 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b73297-1050-4255-bc1d-85845cff79aa admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.661527 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b73297-1050-4255-bc1d-85845cff79aa admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.666195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f07b-92d4-4d49-aef1-79066ec0bde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.666831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f07b-92d4-4d49-aef1-79066ec0bde5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.667110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f07b-92d4-4d49-aef1-79066ec0bde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.667495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4f07b-92d4-4d49-aef1-79066ec0bde5 admin admin] PATH_INFO: `/v3/projects/bdfe09edb3a9418ab22b527879419a29/users/9f024c8a547d4eaab953581c7e7124bc/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.683258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4f07b-92d4-4d49-aef1-79066ec0bde5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdfe09edb3a9418ab22b527879419a29, user_id=9f024c8a547d4eaab953581c7e7124bc, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.685968 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4f07b-92d4-4d49-aef1-79066ec0bde5 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.698287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1241/2470] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:52 2022] PUT /identity/v3/projects/bdfe09edb3a9418ab22b527879419a29/users/9f024c8a547d4eaab953581c7e7124bc/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:52.708977 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c391c5-c775-49bd-86a9-b75e1215ea04 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.709264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c391c5-c775-49bd-86a9-b75e1215ea04 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.709562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c391c5-c775-49bd-86a9-b75e1215ea04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.717868 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:52.738239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-74c391c5-c775-49bd-86a9-b75e1215ea04 None None] MFA Rules not processed for user `9f024c8a547d4eaab953581c7e7124bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:52.739454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-74c391c5-c775-49bd-86a9-b75e1215ea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:52.802217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1230/2471] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:52.806713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f07b-92d4-4d49-aef1-79066ec0bde5 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.869307 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4f07b-92d4-4d49-aef1-79066ec0bde5 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.874497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd04fee-eab7-4ec5-a36b-9acbb634dc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.875099 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd04fee-eab7-4ec5-a36b-9acbb634dc33 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.875376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd04fee-eab7-4ec5-a36b-9acbb634dc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.875650 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd04fee-eab7-4ec5-a36b-9acbb634dc33 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.876283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd04fee-eab7-4ec5-a36b-9acbb634dc33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.878206 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd04fee-eab7-4ec5-a36b-9acbb634dc33 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.907481 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1242/2472] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 13:59:52 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:59:52.911493 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c391c5-c775-49bd-86a9-b75e1215ea04 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.923048 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c391c5-c775-49bd-86a9-b75e1215ea04 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.980330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd0a80-46a5-4483-a9bc-1fc37bad6bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.980938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd0a80-46a5-4483-a9bc-1fc37bad6bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.981216 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd0a80-46a5-4483-a9bc-1fc37bad6bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.981482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd0a80-46a5-4483-a9bc-1fc37bad6bb0 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.982161 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd0a80-46a5-4483-a9bc-1fc37bad6bb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.984074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd0a80-46a5-4483-a9bc-1fc37bad6bb0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:53.006220 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1231/2473] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 13:59:52 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:59:53.010440 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd04fee-eab7-4ec5-a36b-9acbb634dc33 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:53.020824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd04fee-eab7-4ec5-a36b-9acbb634dc33 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:53.025426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc240b-1683-463a-bf90-b82dd6276056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:53.026053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc240b-1683-463a-bf90-b82dd6276056 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:53.026331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc240b-1683-463a-bf90-b82dd6276056 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:53.026635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc240b-1683-463a-bf90-b82dd6276056 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:53.027296 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc240b-1683-463a-bf90-b82dd6276056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:53.028910 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc240b-1683-463a-bf90-b82dd6276056 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:53.039389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1243/2474] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:53.044225 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd0a80-46a5-4483-a9bc-1fc37bad6bb0 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:53.054237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd0a80-46a5-4483-a9bc-1fc37bad6bb0 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:53.059594 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97feabc-ec24-41e6-be26-c0cc83ef6dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:53.060189 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97feabc-ec24-41e6-be26-c0cc83ef6dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:53.060473 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97feabc-ec24-41e6-be26-c0cc83ef6dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:53.060752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97feabc-ec24-41e6-be26-c0cc83ef6dd9 admin admin] PATH_INFO: `/v3/projects/bc3b7d353ba647969f67dce6cfa88279/users/7847f3be51534b579c8afb9597449ce3/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:53.076192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97feabc-ec24-41e6-be26-c0cc83ef6dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3b7d353ba647969f67dce6cfa88279, user_id=7847f3be51534b579c8afb9597449ce3, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:53.078907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97feabc-ec24-41e6-be26-c0cc83ef6dd9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:53.091800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1232/2475] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:53 2022] PUT /identity/v3/projects/bc3b7d353ba647969f67dce6cfa88279/users/7847f3be51534b579c8afb9597449ce3/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:59:53.095747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc240b-1683-463a-bf90-b82dd6276056 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:53.106325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc240b-1683-463a-bf90-b82dd6276056 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:53.134874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:53.164190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2cc7f-c6ca-47d1-8aa7-3ed97a009d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:53.164807 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2cc7f-c6ca-47d1-8aa7-3ed97a009d50 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:53.165084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2cc7f-c6ca-47d1-8aa7-3ed97a009d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:53.165358 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2cc7f-c6ca-47d1-8aa7-3ed97a009d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:53.166071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2cc7f-c6ca-47d1-8aa7-3ed97a009d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:53.168163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2cc7f-c6ca-47d1-8aa7-3ed97a009d50 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:53.178259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1244/2476] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 13:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:59:53.182181 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97feabc-ec24-41e6-be26-c0cc83ef6dd9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:53.192474 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97feabc-ec24-41e6-be26-c0cc83ef6dd9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:53.197110 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1b6fc-d7d0-4ac9-af85-55e1026b9966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:53.197779 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1b6fc-d7d0-4ac9-af85-55e1026b9966 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:53.198063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1b6fc-d7d0-4ac9-af85-55e1026b9966 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:53.198339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1b6fc-d7d0-4ac9-af85-55e1026b9966 admin admin] PATH_INFO: `/v3/projects/bc3b7d353ba647969f67dce6cfa88279/users/7847f3be51534b579c8afb9597449ce3/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:53.200797 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1b6fc-d7d0-4ac9-af85-55e1026b9966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3b7d353ba647969f67dce6cfa88279, user_id=7847f3be51534b579c8afb9597449ce3, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:53.203484 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1b6fc-d7d0-4ac9-af85-55e1026b9966 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:53.213573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1233/2477] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 13:59:53 2022] PUT /identity/v3/projects/bc3b7d353ba647969f67dce6cfa88279/users/7847f3be51534b579c8afb9597449ce3/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:59:53.224454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cb8f0-a4e2-411a-9992-3af1703ff7af None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:53.224741 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cb8f0-a4e2-411a-9992-3af1703ff7af None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:53.225038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cb8f0-a4e2-411a-9992-3af1703ff7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:53.252997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-256cb8f0-a4e2-411a-9992-3af1703ff7af None None] MFA Rules not processed for user `7847f3be51534b579c8afb9597449ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:53.254237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-256cb8f0-a4e2-411a-9992-3af1703ff7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:53.320564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1245/2478] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 13:59:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:53.331787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1b6fc-d7d0-4ac9-af85-55e1026b9966 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:53.333557 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a6e1b6fc-d7d0-4ac9-af85-55e1026b9966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:53.385488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:53.392773 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1b6fc-d7d0-4ac9-af85-55e1026b9966 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:53.396631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdd076-a06d-4ed4-9a66-f0056075fe31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:53.397236 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd076-a06d-4ed4-9a66-f0056075fe31 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:53.397517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd076-a06d-4ed4-9a66-f0056075fe31 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:53.397830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd076-a06d-4ed4-9a66-f0056075fe31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:53.453399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdd076-a06d-4ed4-9a66-f0056075fe31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:53.454926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdd076-a06d-4ed4-9a66-f0056075fe31 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:53.464265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1234/2479] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:57.722235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cb8f0-a4e2-411a-9992-3af1703ff7af None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:57.723899 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-256cb8f0-a4e2-411a-9992-3af1703ff7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:57.736489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cb8f0-a4e2-411a-9992-3af1703ff7af None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:57.740305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5af0f5-e318-4203-b895-5655ef9f1047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:57.740913 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5af0f5-e318-4203-b895-5655ef9f1047 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:57.741189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5af0f5-e318-4203-b895-5655ef9f1047 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:57.741457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5af0f5-e318-4203-b895-5655ef9f1047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:57.751621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5af0f5-e318-4203-b895-5655ef9f1047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:57.753038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5af0f5-e318-4203-b895-5655ef9f1047 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:57.762848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1246/2480] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:02.667931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdd076-a06d-4ed4-9a66-f0056075fe31 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:02.728336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdd076-a06d-4ed4-9a66-f0056075fe31 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:02.733392 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:02.734053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:02.734362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:02.734637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] PATH_INFO: `/v3/users/9f024c8a547d4eaab953581c7e7124bc` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:02.735748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f024c8a547d4eaab953581c7e7124bc)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:02.737676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:02.782577 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f024c8a547d4eaab953581c7e7124bc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:02.789854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f024c8a547d4eaab953581c7e7124bc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:02.790564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f024c8a547d4eaab953581c7e7124bc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:02.791191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f024c8a547d4eaab953581c7e7124bc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:02.798605 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f024c8a547d4eaab953581c7e7124bc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:02.816012 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1235/2481] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:02 2022] DELETE /identity/v3/users/9f024c8a547d4eaab953581c7e7124bc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:02.820427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a80afa-33e9-4f50-9c6c-d526572d9b6f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:02.820906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a80afa-33e9-4f50-9c6c-d526572d9b6f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:02.821019 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a80afa-33e9-4f50-9c6c-d526572d9b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:02.840149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-d6a80afa-33e9-4f50-9c6c-d526572d9b6f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:02.841349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d6a80afa-33e9-4f50-9c6c-d526572d9b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:02.899839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1247/2482] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:02.910217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:02.911708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:02.933247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:02.971565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1441f26-9111-4740-aecc-b64ee7184052 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:02.975239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac79b08-fcec-419d-987b-f722cad13f18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:02.975844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac79b08-fcec-419d-987b-f722cad13f18 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:02.976127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac79b08-fcec-419d-987b-f722cad13f18 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:02.976401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac79b08-fcec-419d-987b-f722cad13f18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:02.985355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac79b08-fcec-419d-987b-f722cad13f18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:02.986820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac79b08-fcec-419d-987b-f722cad13f18 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:02.996695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1236/2483] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:03.269494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a80afa-33e9-4f50-9c6c-d526572d9b6f None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:03.270967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d6a80afa-33e9-4f50-9c6c-d526572d9b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:03.284033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a80afa-33e9-4f50-9c6c-d526572d9b6f None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:03.288882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:03.289587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:03.289996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:03.290341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] PATH_INFO: `/v3/projects/bdfe09edb3a9418ab22b527879419a29` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:03.291665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdfe09edb3a9418ab22b527879419a29)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:03.294318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:03.332441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:03.344017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdfe09edb3a9418ab22b527879419a29', 'request_id': 'req-84bc4781-f29a-4845-9469-d92f879d6bd6'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:03.350287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdfe09edb3a9418ab22b527879419a29', 'request_id': 'req-84bc4781-f29a-4845-9469-d92f879d6bd6'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:03.359160 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdfe09edb3a9418ab22b527879419a29', 'request_id': 'req-84bc4781-f29a-4845-9469-d92f879d6bd6'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:03.369753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] The token cache is being invalidate because project bdfe09edb3a9418ab22b527879419a29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:03.370080 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:03.370701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:03.372194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1248/2484] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:03 2022] DELETE /identity/v3/projects/bdfe09edb3a9418ab22b527879419a29 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:03.376166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac79b08-fcec-419d-987b-f722cad13f18 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:03.386292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac79b08-fcec-419d-987b-f722cad13f18 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:03.449460 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeecdf3-1ea0-4d0a-b548-9869944101a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:03.450055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeecdf3-1ea0-4d0a-b548-9869944101a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:03.450338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeecdf3-1ea0-4d0a-b548-9869944101a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:03.450612 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeecdf3-1ea0-4d0a-b548-9869944101a7 admin admin] PATH_INFO: `/v3/users/7847f3be51534b579c8afb9597449ce3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:03.451748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeecdf3-1ea0-4d0a-b548-9869944101a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7847f3be51534b579c8afb9597449ce3)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:03.453678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeecdf3-1ea0-4d0a-b548-9869944101a7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:03.499254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1eeecdf3-1ea0-4d0a-b548-9869944101a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7847f3be51534b579c8afb9597449ce3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:03.506406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1eeecdf3-1ea0-4d0a-b548-9869944101a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7847f3be51534b579c8afb9597449ce3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:03.507017 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1eeecdf3-1ea0-4d0a-b548-9869944101a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7847f3be51534b579c8afb9597449ce3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:03.507619 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1eeecdf3-1ea0-4d0a-b548-9869944101a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7847f3be51534b579c8afb9597449ce3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:03.515028 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-1eeecdf3-1ea0-4d0a-b548-9869944101a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7847f3be51534b579c8afb9597449ce3'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:03.533091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1237/2485] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:03 2022] DELETE /identity/v3/users/7847f3be51534b579c8afb9597449ce3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:04.061716 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:04.063222 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:04.077835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc4781-f29a-4845-9469-d92f879d6bd6 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:04.144136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53f0672-aaf0-44cb-a9e5-6021aab68337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:04.144909 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f0672-aaf0-44cb-a9e5-6021aab68337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.145305 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f0672-aaf0-44cb-a9e5-6021aab68337 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.145475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53f0672-aaf0-44cb-a9e5-6021aab68337 admin admin] PATH_INFO: `/v3/projects/bc3b7d353ba647969f67dce6cfa88279` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.146697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f0672-aaf0-44cb-a9e5-6021aab68337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc3b7d353ba647969f67dce6cfa88279)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:04.148596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53f0672-aaf0-44cb-a9e5-6021aab68337 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:04.196791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e53f0672-aaf0-44cb-a9e5-6021aab68337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc3b7d353ba647969f67dce6cfa88279', 'request_id': 'req-e53f0672-aaf0-44cb-a9e5-6021aab68337'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:04.203154 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e53f0672-aaf0-44cb-a9e5-6021aab68337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc3b7d353ba647969f67dce6cfa88279', 'request_id': 'req-e53f0672-aaf0-44cb-a9e5-6021aab68337'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:04.217442 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e53f0672-aaf0-44cb-a9e5-6021aab68337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc3b7d353ba647969f67dce6cfa88279', 'request_id': 'req-e53f0672-aaf0-44cb-a9e5-6021aab68337'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:04.228281 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e53f0672-aaf0-44cb-a9e5-6021aab68337 admin admin] The token cache is being invalidate because project bc3b7d353ba647969f67dce6cfa88279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:04.228569 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e53f0672-aaf0-44cb-a9e5-6021aab68337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:04.229184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-e53f0672-aaf0-44cb-a9e5-6021aab68337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:04.231184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1249/2486] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:04 2022] DELETE /identity/v3/projects/bc3b7d353ba647969f67dce6cfa88279 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:04.237404 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cbeca-f8ad-4c3c-a830-aac934dadc15 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.237930 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cbeca-f8ad-4c3c-a830-aac934dadc15 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.238065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cbeca-f8ad-4c3c-a830-aac934dadc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.257548 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-508cbeca-f8ad-4c3c-a830-aac934dadc15 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:04.259386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-508cbeca-f8ad-4c3c-a830-aac934dadc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:04.296288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:04.318647 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1238/2487] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:04.326912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca391d-a492-48c7-b395-beccc8ded2d9 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.327201 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca391d-a492-48c7-b395-beccc8ded2d9 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.327474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca391d-a492-48c7-b395-beccc8ded2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.347609 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-03ca391d-a492-48c7-b395-beccc8ded2d9 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:04.348846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-03ca391d-a492-48c7-b395-beccc8ded2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:04.357341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1250/2488] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:04.361557 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cbeca-f8ad-4c3c-a830-aac934dadc15 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:04.371694 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cbeca-f8ad-4c3c-a830-aac934dadc15 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:04.376226 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500252e-7ad3-42b8-836f-384feabcdcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:04.376823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500252e-7ad3-42b8-836f-384feabcdcaf admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.377100 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500252e-7ad3-42b8-836f-384feabcdcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.377367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500252e-7ad3-42b8-836f-384feabcdcaf admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.378037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1500252e-7ad3-42b8-836f-384feabcdcaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:04.378342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1500252e-7ad3-42b8-836f-384feabcdcaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:04.379931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1500252e-7ad3-42b8-836f-384feabcdcaf admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:04.391177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1239/2489] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:00:04.395404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca391d-a492-48c7-b395-beccc8ded2d9 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:04.404624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca391d-a492-48c7-b395-beccc8ded2d9 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:04.409332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1c930-82c5-42ab-b33c-631e9a810519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:04.409961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1c930-82c5-42ab-b33c-631e9a810519 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.410245 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1c930-82c5-42ab-b33c-631e9a810519 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.410516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1c930-82c5-42ab-b33c-631e9a810519 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.411169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1c930-82c5-42ab-b33c-631e9a810519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:04.413071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc1c930-82c5-42ab-b33c-631e9a810519 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:04.442496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1251/2490] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:04 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:04.446601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500252e-7ad3-42b8-836f-384feabcdcaf admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:04.456389 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500252e-7ad3-42b8-836f-384feabcdcaf admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:04.514722 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfb523-bc10-49a6-a9b4-ab20559380a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:04.515257 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfb523-bc10-49a6-a9b4-ab20559380a1 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.515535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfb523-bc10-49a6-a9b4-ab20559380a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.515881 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bfb523-bc10-49a6-a9b4-ab20559380a1 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.516608 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfb523-bc10-49a6-a9b4-ab20559380a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:04.518754 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bfb523-bc10-49a6-a9b4-ab20559380a1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:04.540886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1240/2491] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:04 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:04.545042 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1c930-82c5-42ab-b33c-631e9a810519 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:04.555400 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1c930-82c5-42ab-b33c-631e9a810519 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:04.560916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803fe89-9202-4d9d-bc3c-c6f437d1def8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:04.561528 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803fe89-9202-4d9d-bc3c-c6f437d1def8 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.561835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803fe89-9202-4d9d-bc3c-c6f437d1def8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.562112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803fe89-9202-4d9d-bc3c-c6f437d1def8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.562790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803fe89-9202-4d9d-bc3c-c6f437d1def8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:04.564451 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803fe89-9202-4d9d-bc3c-c6f437d1def8 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:04.574574 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1252/2492] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:04.578580 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfb523-bc10-49a6-a9b4-ab20559380a1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:04.589213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bfb523-bc10-49a6-a9b4-ab20559380a1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:04.594394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234a296-bcb1-42bf-8d8f-cedb5d249cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:04.594986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234a296-bcb1-42bf-8d8f-cedb5d249cee admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.595266 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234a296-bcb1-42bf-8d8f-cedb5d249cee admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.595566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234a296-bcb1-42bf-8d8f-cedb5d249cee admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.596204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234a296-bcb1-42bf-8d8f-cedb5d249cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:04.597878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234a296-bcb1-42bf-8d8f-cedb5d249cee admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:04.608847 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1241/2493] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:04.612822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803fe89-9202-4d9d-bc3c-c6f437d1def8 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:04.622844 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803fe89-9202-4d9d-bc3c-c6f437d1def8 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:04.627468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078943ee-8111-4a13-bd32-98376c454eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:04.628105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078943ee-8111-4a13-bd32-98376c454eca admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.628385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078943ee-8111-4a13-bd32-98376c454eca admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.628657 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078943ee-8111-4a13-bd32-98376c454eca admin admin] PATH_INFO: `/v3/projects/05b50d4195234880a0cc82e2ab651c77/users/a89e6434cc1a48b19792a6ce7bb14f3d/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.760309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:04.765134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078943ee-8111-4a13-bd32-98376c454eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b50d4195234880a0cc82e2ab651c77, user_id=a89e6434cc1a48b19792a6ce7bb14f3d, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:04.768235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078943ee-8111-4a13-bd32-98376c454eca admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:04.781612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1253/2494] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:04 2022] PUT /identity/v3/projects/05b50d4195234880a0cc82e2ab651c77/users/a89e6434cc1a48b19792a6ce7bb14f3d/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:04.793318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff326-3ba8-40b6-a83d-9b596c805f98 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.793670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff326-3ba8-40b6-a83d-9b596c805f98 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.794047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff326-3ba8-40b6-a83d-9b596c805f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.821475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-ae3ff326-3ba8-40b6-a83d-9b596c805f98 None None] MFA Rules not processed for user `a89e6434cc1a48b19792a6ce7bb14f3d`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:04.822720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ae3ff326-3ba8-40b6-a83d-9b596c805f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:04.886208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1242/2495] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:04.897584 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078943ee-8111-4a13-bd32-98376c454eca admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:04.899336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-078943ee-8111-4a13-bd32-98376c454eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:04.955007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078943ee-8111-4a13-bd32-98376c454eca admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:04.958592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798c02f-5f77-4e0c-9402-63da9ecf6d95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:04.959185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798c02f-5f77-4e0c-9402-63da9ecf6d95 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:04.959465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798c02f-5f77-4e0c-9402-63da9ecf6d95 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:04.959729 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798c02f-5f77-4e0c-9402-63da9ecf6d95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:04.968829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798c02f-5f77-4e0c-9402-63da9ecf6d95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:04.970311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798c02f-5f77-4e0c-9402-63da9ecf6d95 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:04.978993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1254/2496] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:04.997988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ff326-3ba8-40b6-a83d-9b596c805f98 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:05.038701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:05.061611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ff326-3ba8-40b6-a83d-9b596c805f98 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:05.066283 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:05.066903 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:05.067182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:05.067456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] PATH_INFO: `/v3/users/a89e6434cc1a48b19792a6ce7bb14f3d` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:05.068588 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a89e6434cc1a48b19792a6ce7bb14f3d)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:05.070539 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:05.118954 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a89e6434cc1a48b19792a6ce7bb14f3d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:05.126258 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a89e6434cc1a48b19792a6ce7bb14f3d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:05.127085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a89e6434cc1a48b19792a6ce7bb14f3d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:05.127748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a89e6434cc1a48b19792a6ce7bb14f3d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:05.134933 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a89e6434cc1a48b19792a6ce7bb14f3d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:05.152750 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1243/2497] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:04 2022] DELETE /identity/v3/users/a89e6434cc1a48b19792a6ce7bb14f3d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:05.157678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f615ef-b700-412e-8845-8b9c991b5079 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:05.158221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f615ef-b700-412e-8845-8b9c991b5079 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:05.158326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f615ef-b700-412e-8845-8b9c991b5079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:05.176928 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-49f615ef-b700-412e-8845-8b9c991b5079 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:05.178156 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-49f615ef-b700-412e-8845-8b9c991b5079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:05.235531 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1255/2498] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:05.246278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:05.248182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:05.309095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862d2fb-6086-4862-b9d9-e974d6fc5ad8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:05.312595 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b8d9a-f3a7-4a9a-859d-0bd8de108899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:05.313188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b8d9a-f3a7-4a9a-859d-0bd8de108899 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:05.313465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b8d9a-f3a7-4a9a-859d-0bd8de108899 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:05.313744 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b8d9a-f3a7-4a9a-859d-0bd8de108899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:05.322490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b8d9a-f3a7-4a9a-859d-0bd8de108899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:05.324459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b8d9a-f3a7-4a9a-859d-0bd8de108899 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:05.333640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1244/2499] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:05.597269 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f615ef-b700-412e-8845-8b9c991b5079 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:05.598702 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-49f615ef-b700-412e-8845-8b9c991b5079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:05.612602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f615ef-b700-412e-8845-8b9c991b5079 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:05.618768 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:05.619406 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:05.619684 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:05.619999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3 admin admin] PATH_INFO: `/v3/projects/05b50d4195234880a0cc82e2ab651c77` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:05.621684 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b50d4195234880a0cc82e2ab651c77)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:05.623739 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:05.670078 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b50d4195234880a0cc82e2ab651c77', 'request_id': 'req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:05.676373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b50d4195234880a0cc82e2ab651c77', 'request_id': 'req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:05.684282 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b50d4195234880a0cc82e2ab651c77', 'request_id': 'req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:05.694652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3 admin admin] The token cache is being invalidate because project 05b50d4195234880a0cc82e2ab651c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:05.694958 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:05.695577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7fcc1a5f-2558-4ff8-8aa2-5af62402b9a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:05.697051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1256/2500] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:05 2022] DELETE /identity/v3/projects/05b50d4195234880a0cc82e2ab651c77 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:05.703081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cca8c-26e7-4721-a7e8-5fc52eef8518 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:05.703387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cca8c-26e7-4721-a7e8-5fc52eef8518 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:05.703665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11cca8c-26e7-4721-a7e8-5fc52eef8518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:05.723210 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f11cca8c-26e7-4721-a7e8-5fc52eef8518 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:05.724405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f11cca8c-26e7-4721-a7e8-5fc52eef8518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:05.775175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:05.785269 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1245/2501] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:05.795424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddb048-1d4f-46dc-bb91-a0caedd8ce56 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:05.795701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddb048-1d4f-46dc-bb91-a0caedd8ce56 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:05.795970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddb048-1d4f-46dc-bb91-a0caedd8ce56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:05.815577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-09ddb048-1d4f-46dc-bb91-a0caedd8ce56 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:05.816761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-09ddb048-1d4f-46dc-bb91-a0caedd8ce56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:05.824434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1257/2502] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:05.828646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cca8c-26e7-4721-a7e8-5fc52eef8518 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:05.838224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11cca8c-26e7-4721-a7e8-5fc52eef8518 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:05.843039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0c512-00f9-42b0-b422-cc1cceb73c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:05.843579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0c512-00f9-42b0-b422-cc1cceb73c11 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:05.843861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0c512-00f9-42b0-b422-cc1cceb73c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:05.844141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0c512-00f9-42b0-b422-cc1cceb73c11 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:05.844787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0c512-00f9-42b0-b422-cc1cceb73c11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:05.845093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0c512-00f9-42b0-b422-cc1cceb73c11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:05.846728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0c512-00f9-42b0-b422-cc1cceb73c11 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:05.858703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1246/2503] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:00:05.862812 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ddb048-1d4f-46dc-bb91-a0caedd8ce56 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:05.872311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ddb048-1d4f-46dc-bb91-a0caedd8ce56 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:05.876847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4e1bd-343f-42c4-8059-f914ec1d9c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:05.877434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4e1bd-343f-42c4-8059-f914ec1d9c3b admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:05.877711 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4e1bd-343f-42c4-8059-f914ec1d9c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:05.878008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4e1bd-343f-42c4-8059-f914ec1d9c3b admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:05.878639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4e1bd-343f-42c4-8059-f914ec1d9c3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:05.880503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4e1bd-343f-42c4-8059-f914ec1d9c3b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:05.885690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:05.913792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1258/2504] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:05 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:05.968731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0c512-00f9-42b0-b422-cc1cceb73c11 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:05.978958 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0c512-00f9-42b0-b422-cc1cceb73c11 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:06.036283 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6efae-250c-4acc-8f8a-54b85ffff52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:06.036902 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6efae-250c-4acc-8f8a-54b85ffff52a admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:06.037186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6efae-250c-4acc-8f8a-54b85ffff52a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:06.037463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6efae-250c-4acc-8f8a-54b85ffff52a admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:06.038151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6efae-250c-4acc-8f8a-54b85ffff52a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:06.040062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6efae-250c-4acc-8f8a-54b85ffff52a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:06.061373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1247/2505] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:05 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:00:06.065512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4e1bd-343f-42c4-8059-f914ec1d9c3b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:06.076049 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4e1bd-343f-42c4-8059-f914ec1d9c3b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:06.080498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa24d5a-2737-497d-b28f-35cc1370cc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:06.081084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa24d5a-2737-497d-b28f-35cc1370cc0d admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:06.081356 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa24d5a-2737-497d-b28f-35cc1370cc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:06.081621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa24d5a-2737-497d-b28f-35cc1370cc0d admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:06.082293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa24d5a-2737-497d-b28f-35cc1370cc0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:06.083916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa24d5a-2737-497d-b28f-35cc1370cc0d admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:06.093793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1259/2506] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:06.097521 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6efae-250c-4acc-8f8a-54b85ffff52a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:06.107687 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6efae-250c-4acc-8f8a-54b85ffff52a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:06.112496 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5715bdd-be63-4dfe-bb9e-7c28ed76fbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:06.113099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5715bdd-be63-4dfe-bb9e-7c28ed76fbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:06.113383 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5715bdd-be63-4dfe-bb9e-7c28ed76fbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:06.113659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5715bdd-be63-4dfe-bb9e-7c28ed76fbc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:06.114337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5715bdd-be63-4dfe-bb9e-7c28ed76fbc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:06.115957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5715bdd-be63-4dfe-bb9e-7c28ed76fbc2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:06.127144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1248/2507] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:06.131323 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa24d5a-2737-497d-b28f-35cc1370cc0d admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:06.141403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa24d5a-2737-497d-b28f-35cc1370cc0d admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:06.145862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad5a07-6fa7-4f1b-9ab4-f4d39b9218f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:06.146493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad5a07-6fa7-4f1b-9ab4-f4d39b9218f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:06.146767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad5a07-6fa7-4f1b-9ab4-f4d39b9218f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:06.147040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad5a07-6fa7-4f1b-9ab4-f4d39b9218f4 admin admin] PATH_INFO: `/v3/projects/486f92d486e94e1e91b9cef7d7a185eb/users/ced1b34c015f40abb2288fafbe5d841e/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:06.161493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad5a07-6fa7-4f1b-9ab4-f4d39b9218f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486f92d486e94e1e91b9cef7d7a185eb, user_id=ced1b34c015f40abb2288fafbe5d841e, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:06.164192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad5a07-6fa7-4f1b-9ab4-f4d39b9218f4 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:06.177108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1260/2508] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:06 2022] PUT /identity/v3/projects/486f92d486e94e1e91b9cef7d7a185eb/users/ced1b34c015f40abb2288fafbe5d841e/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:06.187405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c084c-ba17-45cf-a1aa-e080ce5e1b02 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:06.187723 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c084c-ba17-45cf-a1aa-e080ce5e1b02 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:06.188000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c084c-ba17-45cf-a1aa-e080ce5e1b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:06.216337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-004c084c-ba17-45cf-a1aa-e080ce5e1b02 None None] MFA Rules not processed for user `ced1b34c015f40abb2288fafbe5d841e`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:06.217545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-004c084c-ba17-45cf-a1aa-e080ce5e1b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:06.280949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1249/2509] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:06.295243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad5a07-6fa7-4f1b-9ab4-f4d39b9218f4 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:06.296962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-3fad5a07-6fa7-4f1b-9ab4-f4d39b9218f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:06.354929 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad5a07-6fa7-4f1b-9ab4-f4d39b9218f4 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:06.358585 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31c50c-d08a-4604-8cd4-8de706d0488c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e27f5b81c704dbebbd6a83b2713e237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:06.359186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31c50c-d08a-4604-8cd4-8de706d0488c service glance] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:06.359467 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31c50c-d08a-4604-8cd4-8de706d0488c service glance] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:06.359738 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31c50c-d08a-4604-8cd4-8de706d0488c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:06.367857 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31c50c-d08a-4604-8cd4-8de706d0488c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:06.369265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31c50c-d08a-4604-8cd4-8de706d0488c service glance] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:06.378437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1261/2510] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:16.278249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:16.374519 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:35.064206 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeaa9d-d083-422d-a156-19c70b6f32f0 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:35.064770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeaa9d-d083-422d-a156-19c70b6f32f0 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:35.064770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeaa9d-d083-422d-a156-19c70b6f32f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:35.120873 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5efeaa9d-d083-422d-a156-19c70b6f32f0 None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:35.122263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5efeaa9d-d083-422d-a156-19c70b6f32f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:35.250971 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1250/2511] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 14:00:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:35.306277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31c50c-d08a-4604-8cd4-8de706d0488c service glance] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:35.308488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-bd31c50c-d08a-4604-8cd4-8de706d0488c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:35.444137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31c50c-d08a-4604-8cd4-8de706d0488c service glance] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:35.454645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e78d8-5717-49e3-a8a1-cbacd18dcbed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:35.455372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e78d8-5717-49e3-a8a1-cbacd18dcbed service swift] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:35.455705 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e78d8-5717-49e3-a8a1-cbacd18dcbed service swift] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:35.456206 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e78d8-5717-49e3-a8a1-cbacd18dcbed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:35.466185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e78d8-5717-49e3-a8a1-cbacd18dcbed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:35.480557 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e78d8-5717-49e3-a8a1-cbacd18dcbed service swift] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:35.525702 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1262/2512] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 14:00:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:00:41.235029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeaa9d-d083-422d-a156-19c70b6f32f0 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:41.236776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5efeaa9d-d083-422d-a156-19c70b6f32f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:41.418224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeaa9d-d083-422d-a156-19c70b6f32f0 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:41.437875 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c75b6f-83f9-476a-8aef-b6328fdbae59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:41.439201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c75b6f-83f9-476a-8aef-b6328fdbae59 service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:41.439472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c75b6f-83f9-476a-8aef-b6328fdbae59 service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:41.439858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c75b6f-83f9-476a-8aef-b6328fdbae59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:41.442377 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-57c75b6f-83f9-476a-8aef-b6328fdbae59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:41.457110 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c75b6f-83f9-476a-8aef-b6328fdbae59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:41.458770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c75b6f-83f9-476a-8aef-b6328fdbae59 service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:41.480687 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1251/2513] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:42.849841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26c22b-8d5e-412c-bf87-18a3f3aa3ede None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:42.850712 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26c22b-8d5e-412c-bf87-18a3f3aa3ede None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:42.851165 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26c22b-8d5e-412c-bf87-18a3f3aa3ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:42.880679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-de26c22b-8d5e-412c-bf87-18a3f3aa3ede None None] MFA Rules not processed for user `76a9dc3ddb144e599a2856fce8dc339d`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:42.882213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-de26c22b-8d5e-412c-bf87-18a3f3aa3ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:43.006038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1263/2514] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:45.076855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c75b6f-83f9-476a-8aef-b6328fdbae59 service placement] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:45.078338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-57c75b6f-83f9-476a-8aef-b6328fdbae59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:45.139887 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:45.236507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c75b6f-83f9-476a-8aef-b6328fdbae59 service placement] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:45.242027 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac71a2-173b-471f-95c1-f116ae8e57ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:45.243060 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac71a2-173b-471f-95c1-f116ae8e57ab service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:45.244357 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac71a2-173b-471f-95c1-f116ae8e57ab service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:45.244989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac71a2-173b-471f-95c1-f116ae8e57ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:45.247198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8cac71a2-173b-471f-95c1-f116ae8e57ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:45.358401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac71a2-173b-471f-95c1-f116ae8e57ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:45.360743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac71a2-173b-471f-95c1-f116ae8e57ab service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:45.390935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1252/2515] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:46.297714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26c22b-8d5e-412c-bf87-18a3f3aa3ede None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:46.310938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de26c22b-8d5e-412c-bf87-18a3f3aa3ede None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:46.315817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:46.316622 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:46.317228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:46.317708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] PATH_INFO: `/v3/users/ced1b34c015f40abb2288fafbe5d841e` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:46.319073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced1b34c015f40abb2288fafbe5d841e)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:46.321441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:46.398974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced1b34c015f40abb2288fafbe5d841e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:46.414039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced1b34c015f40abb2288fafbe5d841e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:46.415307 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced1b34c015f40abb2288fafbe5d841e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:46.416826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced1b34c015f40abb2288fafbe5d841e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:46.423558 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:46.430826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced1b34c015f40abb2288fafbe5d841e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:46.464558 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1264/2516] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:46 2022] DELETE /identity/v3/users/ced1b34c015f40abb2288fafbe5d841e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:46.469514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176eb09-49c2-4488-86ba-034f8f9ee91c None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:46.470343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176eb09-49c2-4488-86ba-034f8f9ee91c None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:46.470612 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4176eb09-49c2-4488-86ba-034f8f9ee91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:46.506341 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4176eb09-49c2-4488-86ba-034f8f9ee91c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:46.508245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4176eb09-49c2-4488-86ba-034f8f9ee91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:46.610267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1253/2517] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:46.621117 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:46.624013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:46.700630 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a0c44-610d-477e-a2c9-bff1e79874c0 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:46.705474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc93225-4ebb-48d7-bce8-8adb69ae5ad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:46.706143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc93225-4ebb-48d7-bce8-8adb69ae5ad4 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:46.706424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc93225-4ebb-48d7-bce8-8adb69ae5ad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:46.706697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc93225-4ebb-48d7-bce8-8adb69ae5ad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:46.725945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc93225-4ebb-48d7-bce8-8adb69ae5ad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:46.727433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc93225-4ebb-48d7-bce8-8adb69ae5ad4 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:46.741794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1265/2518] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:47.266749 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176eb09-49c2-4488-86ba-034f8f9ee91c None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:47.268443 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4176eb09-49c2-4488-86ba-034f8f9ee91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:47.284720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4176eb09-49c2-4488-86ba-034f8f9ee91c None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:47.303510 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:47.304177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:47.304460 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:47.304738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] PATH_INFO: `/v3/projects/486f92d486e94e1e91b9cef7d7a185eb` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:47.307139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=486f92d486e94e1e91b9cef7d7a185eb)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:47.309079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:47.363973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:47.433304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '486f92d486e94e1e91b9cef7d7a185eb', 'request_id': 'req-d244ef4b-600d-49cf-9e49-37add9ec0ac4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:47.450807 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '486f92d486e94e1e91b9cef7d7a185eb', 'request_id': 'req-d244ef4b-600d-49cf-9e49-37add9ec0ac4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:47.457900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '486f92d486e94e1e91b9cef7d7a185eb', 'request_id': 'req-d244ef4b-600d-49cf-9e49-37add9ec0ac4'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:47.498640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] The token cache is being invalidate because project 486f92d486e94e1e91b9cef7d7a185eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:47.499146 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:47.499834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:47.524093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1254/2519] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:47 2022] DELETE /identity/v3/projects/486f92d486e94e1e91b9cef7d7a185eb => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:49.126995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc93225-4ebb-48d7-bce8-8adb69ae5ad4 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:49.129243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-fdc93225-4ebb-48d7-bce8-8adb69ae5ad4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:49.159726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc93225-4ebb-48d7-bce8-8adb69ae5ad4 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:49.221346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e738b-965e-4835-99e9-00c8feb940ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:49.222617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e738b-965e-4835-99e9-00c8feb940ba service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:49.222617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e738b-965e-4835-99e9-00c8feb940ba service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:49.222617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732e738b-965e-4835-99e9-00c8feb940ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:49.225398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-732e738b-965e-4835-99e9-00c8feb940ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:49.302645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e738b-965e-4835-99e9-00c8feb940ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:49.304262 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732e738b-965e-4835-99e9-00c8feb940ba service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:49.314866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1266/2520] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:50.650962 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:50.652512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:50.667536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244ef4b-600d-49cf-9e49-37add9ec0ac4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:50.672750 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636cef0-0bc9-4b0d-b325-e8b593056fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:50.673389 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636cef0-0bc9-4b0d-b325-e8b593056fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:50.673669 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636cef0-0bc9-4b0d-b325-e8b593056fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:50.674305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636cef0-0bc9-4b0d-b325-e8b593056fcf admin admin] PATH_INFO: `/v3/users/2e48a8c1b91142cf9d1af17f28e41961` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:50.675805 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636cef0-0bc9-4b0d-b325-e8b593056fcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e48a8c1b91142cf9d1af17f28e41961)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:50.677079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636cef0-0bc9-4b0d-b325-e8b593056fcf admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:50.731099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9636cef0-0bc9-4b0d-b325-e8b593056fcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e48a8c1b91142cf9d1af17f28e41961'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:50.739967 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9636cef0-0bc9-4b0d-b325-e8b593056fcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e48a8c1b91142cf9d1af17f28e41961'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:50.740839 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9636cef0-0bc9-4b0d-b325-e8b593056fcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e48a8c1b91142cf9d1af17f28e41961'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:50.741605 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9636cef0-0bc9-4b0d-b325-e8b593056fcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e48a8c1b91142cf9d1af17f28e41961'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:50.749255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9636cef0-0bc9-4b0d-b325-e8b593056fcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e48a8c1b91142cf9d1af17f28e41961'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:50.772420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1255/2521] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:50 2022] DELETE /identity/v3/users/2e48a8c1b91142cf9d1af17f28e41961 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:50.950299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e738b-965e-4835-99e9-00c8feb940ba service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:50.968543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732e738b-965e-4835-99e9-00c8feb940ba service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:50.996926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:51.033321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563309e-5448-44f9-b3f8-23dc6703cee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:51.033918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563309e-5448-44f9-b3f8-23dc6703cee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:51.034197 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563309e-5448-44f9-b3f8-23dc6703cee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:51.034495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563309e-5448-44f9-b3f8-23dc6703cee2 admin admin] PATH_INFO: `/v3/projects/b4ca185497284e9780eba8fc2e713d6e` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:51.035617 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563309e-5448-44f9-b3f8-23dc6703cee2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4ca185497284e9780eba8fc2e713d6e)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:51.037522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563309e-5448-44f9-b3f8-23dc6703cee2 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:51.091489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a563309e-5448-44f9-b3f8-23dc6703cee2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4ca185497284e9780eba8fc2e713d6e', 'request_id': 'req-a563309e-5448-44f9-b3f8-23dc6703cee2'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:51.099244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a563309e-5448-44f9-b3f8-23dc6703cee2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4ca185497284e9780eba8fc2e713d6e', 'request_id': 'req-a563309e-5448-44f9-b3f8-23dc6703cee2'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:51.109335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a563309e-5448-44f9-b3f8-23dc6703cee2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4ca185497284e9780eba8fc2e713d6e', 'request_id': 'req-a563309e-5448-44f9-b3f8-23dc6703cee2'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:51.121876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a563309e-5448-44f9-b3f8-23dc6703cee2 admin admin] The token cache is being invalidate because project b4ca185497284e9780eba8fc2e713d6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:51.122276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a563309e-5448-44f9-b3f8-23dc6703cee2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:51.123020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a563309e-5448-44f9-b3f8-23dc6703cee2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:51.130077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7843d-6bdc-49de-b72b-0f8bf391a809 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:51.130515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7843d-6bdc-49de-b72b-0f8bf391a809 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:51.131045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7843d-6bdc-49de-b72b-0f8bf391a809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:51.143112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1267/2522] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:50 2022] DELETE /identity/v3/projects/b4ca185497284e9780eba8fc2e713d6e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:51.191073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-7dd7843d-6bdc-49de-b72b-0f8bf391a809 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:51.192500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-7dd7843d-6bdc-49de-b72b-0f8bf391a809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:51.256146 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:51.267797 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1256/2523] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:51.276713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd398ea-a748-4012-a11d-1c7ba425d4b3 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:51.276998 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd398ea-a748-4012-a11d-1c7ba425d4b3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:51.277302 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd398ea-a748-4012-a11d-1c7ba425d4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:51.298819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4dd398ea-a748-4012-a11d-1c7ba425d4b3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:51.300202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4dd398ea-a748-4012-a11d-1c7ba425d4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:51.308820 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1268/2524] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:51.313056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7843d-6bdc-49de-b72b-0f8bf391a809 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:51.323402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7843d-6bdc-49de-b72b-0f8bf391a809 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:51.328169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f7586-77ea-4a6f-98df-b8fb8f683e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:51.328801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f7586-77ea-4a6f-98df-b8fb8f683e41 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:51.329076 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f7586-77ea-4a6f-98df-b8fb8f683e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:51.329358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f7586-77ea-4a6f-98df-b8fb8f683e41 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:51.330064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f7586-77ea-4a6f-98df-b8fb8f683e41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:51.330371 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f7586-77ea-4a6f-98df-b8fb8f683e41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:51.331987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f7586-77ea-4a6f-98df-b8fb8f683e41 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:51.345616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1257/2525] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:00:51.350113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd398ea-a748-4012-a11d-1c7ba425d4b3 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:51.360163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd398ea-a748-4012-a11d-1c7ba425d4b3 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:51.364885 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefb0fd-419e-4314-ae13-6a092dc1b673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:51.365520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefb0fd-419e-4314-ae13-6a092dc1b673 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:51.365831 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefb0fd-419e-4314-ae13-6a092dc1b673 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:51.366101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebefb0fd-419e-4314-ae13-6a092dc1b673 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:51.366732 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebefb0fd-419e-4314-ae13-6a092dc1b673 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:51.368889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebefb0fd-419e-4314-ae13-6a092dc1b673 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:51.396596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1269/2526] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:51 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:51.401584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f7586-77ea-4a6f-98df-b8fb8f683e41 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:51.412106 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f7586-77ea-4a6f-98df-b8fb8f683e41 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:51.474112 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77940b3-782e-4014-bab0-f555dd115dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:51.474728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77940b3-782e-4014-bab0-f555dd115dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:51.475007 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77940b3-782e-4014-bab0-f555dd115dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:51.475280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77940b3-782e-4014-bab0-f555dd115dd7 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:51.475913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77940b3-782e-4014-bab0-f555dd115dd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:51.477892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77940b3-782e-4014-bab0-f555dd115dd7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:51.504312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1258/2527] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:51 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:51.505662 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefb0fd-419e-4314-ae13-6a092dc1b673 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:51.520038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebefb0fd-419e-4314-ae13-6a092dc1b673 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:51.524785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd32de-b3c1-43a3-be52-7f5c29974586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:51.526057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd32de-b3c1-43a3-be52-7f5c29974586 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:51.526404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd32de-b3c1-43a3-be52-7f5c29974586 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:51.526747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd32de-b3c1-43a3-be52-7f5c29974586 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:51.527462 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd32de-b3c1-43a3-be52-7f5c29974586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:51.529111 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd32de-b3c1-43a3-be52-7f5c29974586 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:51.539338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1270/2528] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:00:51.543276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77940b3-782e-4014-bab0-f555dd115dd7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:51.554886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77940b3-782e-4014-bab0-f555dd115dd7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:51.559663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab053d2-3f23-4b53-8fb9-5e309cf680e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:51.560281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab053d2-3f23-4b53-8fb9-5e309cf680e0 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:51.560563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab053d2-3f23-4b53-8fb9-5e309cf680e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:51.560835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab053d2-3f23-4b53-8fb9-5e309cf680e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:51.561481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab053d2-3f23-4b53-8fb9-5e309cf680e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:51.563110 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab053d2-3f23-4b53-8fb9-5e309cf680e0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:51.574771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1259/2529] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:51.578803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd32de-b3c1-43a3-be52-7f5c29974586 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:51.589338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd32de-b3c1-43a3-be52-7f5c29974586 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:51.594477 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d88ddc-f5d3-4bbc-bbc8-5b5f7104d4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:51.595138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d88ddc-f5d3-4bbc-bbc8-5b5f7104d4cb admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:51.595420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d88ddc-f5d3-4bbc-bbc8-5b5f7104d4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:51.595694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d88ddc-f5d3-4bbc-bbc8-5b5f7104d4cb admin admin] PATH_INFO: `/v3/projects/16bdb7cef22847fc8e05717b28bbedb4/users/66f89777d8d54397884aa1d553e921c4/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:51.610975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d88ddc-f5d3-4bbc-bbc8-5b5f7104d4cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=16bdb7cef22847fc8e05717b28bbedb4, user_id=66f89777d8d54397884aa1d553e921c4, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:51.613623 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d88ddc-f5d3-4bbc-bbc8-5b5f7104d4cb admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:51.628037 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1271/2530] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:51 2022] PUT /identity/v3/projects/16bdb7cef22847fc8e05717b28bbedb4/users/66f89777d8d54397884aa1d553e921c4/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:51.640016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ae5e6-8a9a-4109-8408-566c0a1f60a3 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:51.640299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ae5e6-8a9a-4109-8408-566c0a1f60a3 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:51.640597 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ae5e6-8a9a-4109-8408-566c0a1f60a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:51.671115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-545ae5e6-8a9a-4109-8408-566c0a1f60a3 None None] MFA Rules not processed for user `66f89777d8d54397884aa1d553e921c4`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:51.672464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-545ae5e6-8a9a-4109-8408-566c0a1f60a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:51.750012 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1260/2531] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:51.755231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d88ddc-f5d3-4bbc-bbc8-5b5f7104d4cb admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:51.819392 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:51.826883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d88ddc-f5d3-4bbc-bbc8-5b5f7104d4cb admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:51.833292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8d36d-3560-47b9-8152-4e7071e83842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:51.834047 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8d36d-3560-47b9-8152-4e7071e83842 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:51.834420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8d36d-3560-47b9-8152-4e7071e83842 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:51.834784 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8d36d-3560-47b9-8152-4e7071e83842 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:51.835697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8d36d-3560-47b9-8152-4e7071e83842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:51.838413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8d36d-3560-47b9-8152-4e7071e83842 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:51.873275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1272/2532] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:51 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:51.877137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ae5e6-8a9a-4109-8408-566c0a1f60a3 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:51.889402 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ae5e6-8a9a-4109-8408-566c0a1f60a3 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:51.943933 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:51.955215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4290ae-1e6e-4d09-aa9e-89ea99ca3b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:51.955830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4290ae-1e6e-4d09-aa9e-89ea99ca3b81 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:51.956112 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4290ae-1e6e-4d09-aa9e-89ea99ca3b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:51.956386 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4290ae-1e6e-4d09-aa9e-89ea99ca3b81 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:51.957478 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4290ae-1e6e-4d09-aa9e-89ea99ca3b81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:51.960058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4290ae-1e6e-4d09-aa9e-89ea99ca3b81 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:52.003691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1261/2533] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:51 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:52.003846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8d36d-3560-47b9-8152-4e7071e83842 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:52.012566 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8d36d-3560-47b9-8152-4e7071e83842 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:52.019790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c90778-dbe4-4617-9a4d-573551c576e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:52.020534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c90778-dbe4-4617-9a4d-573551c576e9 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:52.020818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c90778-dbe4-4617-9a4d-573551c576e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:52.021723 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c90778-dbe4-4617-9a4d-573551c576e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:52.022750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c90778-dbe4-4617-9a4d-573551c576e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:52.024525 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c90778-dbe4-4617-9a4d-573551c576e9 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:52.036298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1273/2534] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:52.040601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4290ae-1e6e-4d09-aa9e-89ea99ca3b81 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:52.051905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4290ae-1e6e-4d09-aa9e-89ea99ca3b81 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:52.057943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf02f3e-6665-44b7-8be8-74fda576f3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:52.058763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf02f3e-6665-44b7-8be8-74fda576f3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:52.059143 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf02f3e-6665-44b7-8be8-74fda576f3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:52.059519 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf02f3e-6665-44b7-8be8-74fda576f3f7 admin admin] PATH_INFO: `/v3/projects/b1a4083f55834b39a7d6f6487b7d187a/users/4c32d9ace6144d84bf511c82e7bc0ff6/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:52.080615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf02f3e-6665-44b7-8be8-74fda576f3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1a4083f55834b39a7d6f6487b7d187a, user_id=4c32d9ace6144d84bf511c82e7bc0ff6, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:52.083832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf02f3e-6665-44b7-8be8-74fda576f3f7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:52.098379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1262/2535] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:52 2022] PUT /identity/v3/projects/b1a4083f55834b39a7d6f6487b7d187a/users/4c32d9ace6144d84bf511c82e7bc0ff6/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:52.104063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c90778-dbe4-4617-9a4d-573551c576e9 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:52.140640 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c90778-dbe4-4617-9a4d-573551c576e9 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:52.211133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7120c-ded4-4b4f-a416-89ff8f2d7980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:52.211742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7120c-ded4-4b4f-a416-89ff8f2d7980 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:52.212013 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7120c-ded4-4b4f-a416-89ff8f2d7980 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:52.212453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7120c-ded4-4b4f-a416-89ff8f2d7980 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:52.213354 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7120c-ded4-4b4f-a416-89ff8f2d7980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:52.215832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b7120c-ded4-4b4f-a416-89ff8f2d7980 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:52.230480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1274/2536] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:52.234483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf02f3e-6665-44b7-8be8-74fda576f3f7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:52.246503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf02f3e-6665-44b7-8be8-74fda576f3f7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:52.251717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49c58a-9b6d-4b9f-ad33-3923a9b7d06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:52.252383 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49c58a-9b6d-4b9f-ad33-3923a9b7d06e admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:52.252666 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49c58a-9b6d-4b9f-ad33-3923a9b7d06e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:52.252961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49c58a-9b6d-4b9f-ad33-3923a9b7d06e admin admin] PATH_INFO: `/v3/projects/b1a4083f55834b39a7d6f6487b7d187a/users/4c32d9ace6144d84bf511c82e7bc0ff6/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:52.254945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49c58a-9b6d-4b9f-ad33-3923a9b7d06e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1a4083f55834b39a7d6f6487b7d187a, user_id=4c32d9ace6144d84bf511c82e7bc0ff6, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:52.257625 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49c58a-9b6d-4b9f-ad33-3923a9b7d06e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:52.269470 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1263/2537] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:52 2022] PUT /identity/v3/projects/b1a4083f55834b39a7d6f6487b7d187a/users/4c32d9ace6144d84bf511c82e7bc0ff6/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:52.280421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee75a4-c886-4286-8737-0637a10d9036 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:52.281495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee75a4-c886-4286-8737-0637a10d9036 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:52.282140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfee75a4-c886-4286-8737-0637a10d9036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:52.318924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-bfee75a4-c886-4286-8737-0637a10d9036 None None] MFA Rules not processed for user `4c32d9ace6144d84bf511c82e7bc0ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:52.320226 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-bfee75a4-c886-4286-8737-0637a10d9036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:52.424612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1275/2538] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:52.436468 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49c58a-9b6d-4b9f-ad33-3923a9b7d06e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:52.438420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-ce49c58a-9b6d-4b9f-ad33-3923a9b7d06e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:52.538740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49c58a-9b6d-4b9f-ad33-3923a9b7d06e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:52.542797 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aefb8-fc32-4016-a03a-21206aaac34f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:52.543420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aefb8-fc32-4016-a03a-21206aaac34f service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:52.543699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aefb8-fc32-4016-a03a-21206aaac34f service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:52.543971 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aefb8-fc32-4016-a03a-21206aaac34f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:52.554733 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aefb8-fc32-4016-a03a-21206aaac34f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:52.556945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aefb8-fc32-4016-a03a-21206aaac34f service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:52.567783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1264/2539] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:52.654949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee75a4-c886-4286-8737-0637a10d9036 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:52.665077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfee75a4-c886-4286-8737-0637a10d9036 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:52.670059 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bbea6-cc8d-456d-81d6-02b8eb54c4a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:52.670764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bbea6-cc8d-456d-81d6-02b8eb54c4a4 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:52.671107 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bbea6-cc8d-456d-81d6-02b8eb54c4a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:52.671671 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bbea6-cc8d-456d-81d6-02b8eb54c4a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:52.741484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:52.779352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bbea6-cc8d-456d-81d6-02b8eb54c4a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:52.783411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bbea6-cc8d-456d-81d6-02b8eb54c4a4 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:52.808277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1276/2540] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:54.445267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca9595-fc08-455e-b085-408edfbff85f None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:54.445895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca9595-fc08-455e-b085-408edfbff85f None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:54.445895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca9595-fc08-455e-b085-408edfbff85f None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:54.448244 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1265/2541] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 14:00:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:00:54.454384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7d400-866d-4e05-8594-bde02ae4d50f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:54.454908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7d400-866d-4e05-8594-bde02ae4d50f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:54.454908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7d400-866d-4e05-8594-bde02ae4d50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:54.566679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-33d7d400-866d-4e05-8594-bde02ae4d50f None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:54.570987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-33d7d400-866d-4e05-8594-bde02ae4d50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:54.713918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1277/2542] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 14:00:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:54.726528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca9595-fc08-455e-b085-408edfbff85f None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:54.729066 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-86ca9595-fc08-455e-b085-408edfbff85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:54.793096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca9595-fc08-455e-b085-408edfbff85f None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:54.796601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f997b02f-b367-4f41-a1cd-5bfbd0cc3b82 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e27f5b81c704dbebbd6a83b2713e237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:54.797205 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997b02f-b367-4f41-a1cd-5bfbd0cc3b82 service glance] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:54.797480 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997b02f-b367-4f41-a1cd-5bfbd0cc3b82 service glance] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:54.797774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997b02f-b367-4f41-a1cd-5bfbd0cc3b82 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:54.808182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997b02f-b367-4f41-a1cd-5bfbd0cc3b82 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:54.809598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f997b02f-b367-4f41-a1cd-5bfbd0cc3b82 service glance] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:54.820470 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1266/2543] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:54.893464 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd77575-3326-4c2c-9c17-b943fe1dc23a None None] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:54.893803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd77575-3326-4c2c-9c17-b943fe1dc23a None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:54.894119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd77575-3326-4c2c-9c17-b943fe1dc23a None None] PATH_INFO: `/` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:54.895738 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1278/2544] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 14:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:00:54.901326 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fad69-4a51-4e7a-a77c-d179c11c6712 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:54.901761 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fad69-4a51-4e7a-a77c-d179c11c6712 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:54.902090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fad69-4a51-4e7a-a77c-d179c11c6712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:54.918028 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd77575-3326-4c2c-9c17-b943fe1dc23a None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:54.919911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-afd77575-3326-4c2c-9c17-b943fe1dc23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:54.924164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-670fad69-4a51-4e7a-a77c-d179c11c6712 None None] MFA Rules not processed for user `531ebc79bf424e9b8762b2c7ec7fe8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:54.925331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-670fad69-4a51-4e7a-a77c-d179c11c6712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:54.954465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:54.986999 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1267/2545] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 14:00:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:54.994434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd77575-3326-4c2c-9c17-b943fe1dc23a None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:54.998725 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fad69-4a51-4e7a-a77c-d179c11c6712 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:54.999541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33e548-f3e8-4b6c-a5d8-c5e77035e21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:55.000162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-670fad69-4a51-4e7a-a77c-d179c11c6712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:55.000284 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33e548-f3e8-4b6c-a5d8-c5e77035e21a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:55.000457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33e548-f3e8-4b6c-a5d8-c5e77035e21a admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:55.000725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33e548-f3e8-4b6c-a5d8-c5e77035e21a admin admin] PATH_INFO: `/v3/users/96744eb6a9cf4895b6101e209d1fdd29` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:55.001878 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33e548-f3e8-4b6c-a5d8-c5e77035e21a admin admin] RBAC: Authorizing `identity:delete_user(user_id=96744eb6a9cf4895b6101e209d1fdd29)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:55.003765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33e548-f3e8-4b6c-a5d8-c5e77035e21a admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:55.011009 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fad69-4a51-4e7a-a77c-d179c11c6712 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:55.014690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729743e-a916-4b89-8055-4e68340eecdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:55.015292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729743e-a916-4b89-8055-4e68340eecdf service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:55.015573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729743e-a916-4b89-8055-4e68340eecdf service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:55.016243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729743e-a916-4b89-8055-4e68340eecdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:55.023088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729743e-a916-4b89-8055-4e68340eecdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:55.024503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729743e-a916-4b89-8055-4e68340eecdf service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:55.032900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1268/2546] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:00:55.052670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bc33e548-f3e8-4b6c-a5d8-c5e77035e21a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96744eb6a9cf4895b6101e209d1fdd29'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:55.061227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bc33e548-f3e8-4b6c-a5d8-c5e77035e21a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96744eb6a9cf4895b6101e209d1fdd29'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:55.062983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bc33e548-f3e8-4b6c-a5d8-c5e77035e21a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96744eb6a9cf4895b6101e209d1fdd29'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:55.063745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bc33e548-f3e8-4b6c-a5d8-c5e77035e21a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96744eb6a9cf4895b6101e209d1fdd29'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:55.072336 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bc33e548-f3e8-4b6c-a5d8-c5e77035e21a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96744eb6a9cf4895b6101e209d1fdd29'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:55.073612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:55.094213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1279/2547] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:54 2022] DELETE /identity/v3/users/96744eb6a9cf4895b6101e209d1fdd29 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:55.272143 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729743e-a916-4b89-8055-4e68340eecdf service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:55.289401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729743e-a916-4b89-8055-4e68340eecdf service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:55.354676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc3c3d-0d72-41d2-aad9-1e821fc574da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:55.355332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc3c3d-0d72-41d2-aad9-1e821fc574da admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:55.355605 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc3c3d-0d72-41d2-aad9-1e821fc574da admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:55.355894 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc3c3d-0d72-41d2-aad9-1e821fc574da admin admin] PATH_INFO: `/v3/projects/6998e0c9a62e4f54a27faf56dbeb543a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:55.357008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc3c3d-0d72-41d2-aad9-1e821fc574da admin admin] RBAC: Authorizing `identity:delete_project(project_id=6998e0c9a62e4f54a27faf56dbeb543a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:55.359020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc3c3d-0d72-41d2-aad9-1e821fc574da admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:55.407208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-04fc3c3d-0d72-41d2-aad9-1e821fc574da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6998e0c9a62e4f54a27faf56dbeb543a', 'request_id': 'req-04fc3c3d-0d72-41d2-aad9-1e821fc574da'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:55.413783 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-04fc3c3d-0d72-41d2-aad9-1e821fc574da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6998e0c9a62e4f54a27faf56dbeb543a', 'request_id': 'req-04fc3c3d-0d72-41d2-aad9-1e821fc574da'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:55.419942 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-04fc3c3d-0d72-41d2-aad9-1e821fc574da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6998e0c9a62e4f54a27faf56dbeb543a', 'request_id': 'req-04fc3c3d-0d72-41d2-aad9-1e821fc574da'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:55.433794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-04fc3c3d-0d72-41d2-aad9-1e821fc574da admin admin] The token cache is being invalidate because project 6998e0c9a62e4f54a27faf56dbeb543a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:55.434201 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-04fc3c3d-0d72-41d2-aad9-1e821fc574da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:55.434916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-04fc3c3d-0d72-41d2-aad9-1e821fc574da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:55.436476 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1269/2548] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:55 2022] DELETE /identity/v3/projects/6998e0c9a62e4f54a27faf56dbeb543a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:55.441571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb826d0-0517-4ddf-b82e-0c9d1d0a874b None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:55.441888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb826d0-0517-4ddf-b82e-0c9d1d0a874b None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:55.442162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb826d0-0517-4ddf-b82e-0c9d1d0a874b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:55.463215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-ccb826d0-0517-4ddf-b82e-0c9d1d0a874b None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:55.464469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ccb826d0-0517-4ddf-b82e-0c9d1d0a874b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:55.529850 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1280/2549] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:55.543401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f723a4-9423-4ead-8045-5c0d925b4986 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:55.543831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f723a4-9423-4ead-8045-5c0d925b4986 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:55.543957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f723a4-9423-4ead-8045-5c0d925b4986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:55.569381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a8f723a4-9423-4ead-8045-5c0d925b4986 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:55.570881 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a8f723a4-9423-4ead-8045-5c0d925b4986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:55.581155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1270/2550] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:55.586392 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb826d0-0517-4ddf-b82e-0c9d1d0a874b None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:55.600164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb826d0-0517-4ddf-b82e-0c9d1d0a874b None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:55.606268 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920a66-c2be-428a-86a7-caa9eb1ae714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:55.607388 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920a66-c2be-428a-86a7-caa9eb1ae714 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:55.607533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920a66-c2be-428a-86a7-caa9eb1ae714 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:55.607814 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920a66-c2be-428a-86a7-caa9eb1ae714 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:55.608540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920a66-c2be-428a-86a7-caa9eb1ae714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:55.608846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920a66-c2be-428a-86a7-caa9eb1ae714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:55.610506 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920a66-c2be-428a-86a7-caa9eb1ae714 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:55.625792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1281/2551] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:00:55.627739 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f723a4-9423-4ead-8045-5c0d925b4986 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:55.639609 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f723a4-9423-4ead-8045-5c0d925b4986 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:55.645915 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078cfe3-b6b6-42cf-8c64-22c03c68bbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:55.646640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078cfe3-b6b6-42cf-8c64-22c03c68bbdb admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:55.646985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078cfe3-b6b6-42cf-8c64-22c03c68bbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:55.647328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078cfe3-b6b6-42cf-8c64-22c03c68bbdb admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:55.648056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078cfe3-b6b6-42cf-8c64-22c03c68bbdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:55.650589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078cfe3-b6b6-42cf-8c64-22c03c68bbdb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:55.698409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1271/2552] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:55 2022] POST /identity/v3/projects => generated 365 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:55.699672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920a66-c2be-428a-86a7-caa9eb1ae714 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:55.710208 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920a66-c2be-428a-86a7-caa9eb1ae714 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:55.814564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66272b1-0899-4027-80d6-4e6a2d0687fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:55.815450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66272b1-0899-4027-80d6-4e6a2d0687fa admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:55.815825 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66272b1-0899-4027-80d6-4e6a2d0687fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:55.816221 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66272b1-0899-4027-80d6-4e6a2d0687fa admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:55.817020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66272b1-0899-4027-80d6-4e6a2d0687fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:55.819423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66272b1-0899-4027-80d6-4e6a2d0687fa admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:55.851704 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1282/2553] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:55 2022] POST /identity/v3/users => generated 283 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:55.857005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078cfe3-b6b6-42cf-8c64-22c03c68bbdb admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:55.869510 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078cfe3-b6b6-42cf-8c64-22c03c68bbdb admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:55.876171 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848b82f-1f08-4aeb-a033-a43e46a45a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:55.877224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848b82f-1f08-4aeb-a033-a43e46a45a0a admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:55.877828 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848b82f-1f08-4aeb-a033-a43e46a45a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:55.878262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848b82f-1f08-4aeb-a033-a43e46a45a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:55.879089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848b82f-1f08-4aeb-a033-a43e46a45a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:55.880927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848b82f-1f08-4aeb-a033-a43e46a45a0a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:55.902351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1272/2554] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:55.905313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66272b1-0899-4027-80d6-4e6a2d0687fa admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:55.926837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66272b1-0899-4027-80d6-4e6a2d0687fa admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:55.935084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74e31e-478c-4eef-95d5-8f3b1ba3b581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:55.938168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74e31e-478c-4eef-95d5-8f3b1ba3b581 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:55.938771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74e31e-478c-4eef-95d5-8f3b1ba3b581 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:55.939213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74e31e-478c-4eef-95d5-8f3b1ba3b581 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:55.940753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74e31e-478c-4eef-95d5-8f3b1ba3b581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:55.942800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74e31e-478c-4eef-95d5-8f3b1ba3b581 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:55.962043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1283/2555] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:55.966518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848b82f-1f08-4aeb-a033-a43e46a45a0a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:55.978710 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848b82f-1f08-4aeb-a033-a43e46a45a0a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:55.984475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57834326-91f1-48db-9f66-eb65af7ca06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:55.985423 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57834326-91f1-48db-9f66-eb65af7ca06a admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:55.985708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57834326-91f1-48db-9f66-eb65af7ca06a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:55.986013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57834326-91f1-48db-9f66-eb65af7ca06a admin admin] PATH_INFO: `/v3/projects/795dd49fb5c84a85b8d318c41452f9dc/users/4acfe8cb64f841c3b168bb2e0da43e81/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:56.004086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57834326-91f1-48db-9f66-eb65af7ca06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=795dd49fb5c84a85b8d318c41452f9dc, user_id=4acfe8cb64f841c3b168bb2e0da43e81, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:56.006832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57834326-91f1-48db-9f66-eb65af7ca06a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:56.009273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:56.023149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1273/2556] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:55 2022] PUT /identity/v3/projects/795dd49fb5c84a85b8d318c41452f9dc/users/4acfe8cb64f841c3b168bb2e0da43e81/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:00:56.033492 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78077e5b-3e04-4b5b-9a73-9640ca647f0c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:56.033803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78077e5b-3e04-4b5b-9a73-9640ca647f0c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:56.034079 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78077e5b-3e04-4b5b-9a73-9640ca647f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:56.063701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-78077e5b-3e04-4b5b-9a73-9640ca647f0c None None] MFA Rules not processed for user `4acfe8cb64f841c3b168bb2e0da43e81`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:56.064883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-78077e5b-3e04-4b5b-9a73-9640ca647f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:56.096621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:56.129461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1284/2557] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:56.134174 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57834326-91f1-48db-9f66-eb65af7ca06a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:56.196916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57834326-91f1-48db-9f66-eb65af7ca06a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:56.201537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8303dfa-0fed-4448-8c17-724ce37499ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:56.202214 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8303dfa-0fed-4448-8c17-724ce37499ad admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:56.202636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8303dfa-0fed-4448-8c17-724ce37499ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:56.202911 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8303dfa-0fed-4448-8c17-724ce37499ad admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:56.203559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8303dfa-0fed-4448-8c17-724ce37499ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:56.205537 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8303dfa-0fed-4448-8c17-724ce37499ad admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:56.234391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1274/2558] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:56 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:56.238212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78077e5b-3e04-4b5b-9a73-9640ca647f0c None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:56.248746 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78077e5b-3e04-4b5b-9a73-9640ca647f0c None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:56.303563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cdd5b-3452-4a17-b510-2aee61555abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:56.304168 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cdd5b-3452-4a17-b510-2aee61555abb admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:56.304448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cdd5b-3452-4a17-b510-2aee61555abb admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:56.304718 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cdd5b-3452-4a17-b510-2aee61555abb admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:56.305338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cdd5b-3452-4a17-b510-2aee61555abb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:56.307530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cdd5b-3452-4a17-b510-2aee61555abb admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:56.328214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1285/2559] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:56 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:56.332298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8303dfa-0fed-4448-8c17-724ce37499ad admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:56.343210 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8303dfa-0fed-4448-8c17-724ce37499ad admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:56.347794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc3d7c-551e-4764-ac08-9cd52b113336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:56.348380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc3d7c-551e-4764-ac08-9cd52b113336 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:56.348657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc3d7c-551e-4764-ac08-9cd52b113336 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:56.348926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc3d7c-551e-4764-ac08-9cd52b113336 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:56.349558 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc3d7c-551e-4764-ac08-9cd52b113336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:56.351172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc3d7c-551e-4764-ac08-9cd52b113336 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:56.362040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1275/2560] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:56.366045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cdd5b-3452-4a17-b510-2aee61555abb admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:56.375936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cdd5b-3452-4a17-b510-2aee61555abb admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:56.380537 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209ee2c-0494-4005-a91a-0a82fcb7f514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:56.381191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209ee2c-0494-4005-a91a-0a82fcb7f514 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:56.381457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209ee2c-0494-4005-a91a-0a82fcb7f514 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:56.381748 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209ee2c-0494-4005-a91a-0a82fcb7f514 admin admin] PATH_INFO: `/v3/projects/ba275451ff584b079dd9c277472845e1/users/e6d44706e5404ed89ad1651d82034eab/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:56.396018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209ee2c-0494-4005-a91a-0a82fcb7f514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba275451ff584b079dd9c277472845e1, user_id=e6d44706e5404ed89ad1651d82034eab, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:56.398672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209ee2c-0494-4005-a91a-0a82fcb7f514 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:56.411988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1286/2561] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:56 2022] PUT /identity/v3/projects/ba275451ff584b079dd9c277472845e1/users/e6d44706e5404ed89ad1651d82034eab/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:56.416508 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc3d7c-551e-4764-ac08-9cd52b113336 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:56.426209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc3d7c-551e-4764-ac08-9cd52b113336 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:56.481380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d2270-9926-441b-ae20-086a8d0a7a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:56.482068 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d2270-9926-441b-ae20-086a8d0a7a6e admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:56.482339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d2270-9926-441b-ae20-086a8d0a7a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:56.482599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d2270-9926-441b-ae20-086a8d0a7a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:56.483362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d2270-9926-441b-ae20-086a8d0a7a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:56.485093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d2270-9926-441b-ae20-086a8d0a7a6e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:56.495377 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1276/2562] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:56.499547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209ee2c-0494-4005-a91a-0a82fcb7f514 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:56.509535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209ee2c-0494-4005-a91a-0a82fcb7f514 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:56.514462 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004eb27-c23a-4ce1-9ddf-f090ea7a1f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:56.515140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004eb27-c23a-4ce1-9ddf-f090ea7a1f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:56.515439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004eb27-c23a-4ce1-9ddf-f090ea7a1f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:56.515738 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004eb27-c23a-4ce1-9ddf-f090ea7a1f47 admin admin] PATH_INFO: `/v3/projects/ba275451ff584b079dd9c277472845e1/users/e6d44706e5404ed89ad1651d82034eab/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:56.517665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004eb27-c23a-4ce1-9ddf-f090ea7a1f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba275451ff584b079dd9c277472845e1, user_id=e6d44706e5404ed89ad1651d82034eab, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:56.520313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004eb27-c23a-4ce1-9ddf-f090ea7a1f47 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:56.530987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1287/2563] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:56 2022] PUT /identity/v3/projects/ba275451ff584b079dd9c277472845e1/users/e6d44706e5404ed89ad1651d82034eab/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:56.543016 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53a31e-2056-4a70-8fac-4ca462da2d18 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:56.543305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53a31e-2056-4a70-8fac-4ca462da2d18 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:56.543580 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53a31e-2056-4a70-8fac-4ca462da2d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:56.573133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-dc53a31e-2056-4a70-8fac-4ca462da2d18 None None] MFA Rules not processed for user `e6d44706e5404ed89ad1651d82034eab`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:56.574367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dc53a31e-2056-4a70-8fac-4ca462da2d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:56.623881 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:56.643446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1277/2564] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:56.655703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004eb27-c23a-4ce1-9ddf-f090ea7a1f47 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:56.657459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c004eb27-c23a-4ce1-9ddf-f090ea7a1f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:56.713432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004eb27-c23a-4ce1-9ddf-f090ea7a1f47 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:56.717383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42080a0-11d6-4459-84b2-9287f682cfbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:56.718123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42080a0-11d6-4459-84b2-9287f682cfbe service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:56.718404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42080a0-11d6-4459-84b2-9287f682cfbe service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:56.718676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42080a0-11d6-4459-84b2-9287f682cfbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:56.727553 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42080a0-11d6-4459-84b2-9287f682cfbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:56.728931 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42080a0-11d6-4459-84b2-9287f682cfbe service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:56.738218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1288/2565] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:57.175740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53a31e-2056-4a70-8fac-4ca462da2d18 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:57.238855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53a31e-2056-4a70-8fac-4ca462da2d18 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:57.243281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:57.244424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:57.244710 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:57.245011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] PATH_INFO: `/v3/users/4acfe8cb64f841c3b168bb2e0da43e81` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:57.246152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4acfe8cb64f841c3b168bb2e0da43e81)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:57.248104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:57.292971 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4acfe8cb64f841c3b168bb2e0da43e81'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:57.300097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4acfe8cb64f841c3b168bb2e0da43e81'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:57.300739 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4acfe8cb64f841c3b168bb2e0da43e81'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:57.301340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4acfe8cb64f841c3b168bb2e0da43e81'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:57.308840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4acfe8cb64f841c3b168bb2e0da43e81'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:57.326354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1278/2566] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:57 2022] DELETE /identity/v3/users/4acfe8cb64f841c3b168bb2e0da43e81 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:57.330915 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be1c6b-ca60-42e5-8307-3b777c44eff7 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:57.331195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be1c6b-ca60-42e5-8307-3b777c44eff7 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:57.331461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be1c6b-ca60-42e5-8307-3b777c44eff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:57.349937 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-e1be1c6b-ca60-42e5-8307-3b777c44eff7 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:57.351106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e1be1c6b-ca60-42e5-8307-3b777c44eff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:57.377005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:57.409888 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1289/2567] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:57.419976 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:57.421399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:57.478587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd052448-e8c9-4181-97a1-bb7c64a47735 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:57.482382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d775b-9e16-4a4d-b527-d12710943e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:57.482997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d775b-9e16-4a4d-b527-d12710943e2b service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:57.483303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d775b-9e16-4a4d-b527-d12710943e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:57.483576 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d775b-9e16-4a4d-b527-d12710943e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:57.492572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d775b-9e16-4a4d-b527-d12710943e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:57.494539 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d775b-9e16-4a4d-b527-d12710943e2b service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:57.503887 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1279/2568] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:57.766742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be1c6b-ca60-42e5-8307-3b777c44eff7 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:57.768320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e1be1c6b-ca60-42e5-8307-3b777c44eff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:57.781331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be1c6b-ca60-42e5-8307-3b777c44eff7 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:57.785989 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:57.786627 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:57.786907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:57.787195 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] PATH_INFO: `/v3/projects/795dd49fb5c84a85b8d318c41452f9dc` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:57.788327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] RBAC: Authorizing `identity:delete_project(project_id=795dd49fb5c84a85b8d318c41452f9dc)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:57.790285 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:57.836776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '795dd49fb5c84a85b8d318c41452f9dc', 'request_id': 'req-f328691f-0511-4d0c-b8bb-bdeba94a59be'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:57.843805 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '795dd49fb5c84a85b8d318c41452f9dc', 'request_id': 'req-f328691f-0511-4d0c-b8bb-bdeba94a59be'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:57.851101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '795dd49fb5c84a85b8d318c41452f9dc', 'request_id': 'req-f328691f-0511-4d0c-b8bb-bdeba94a59be'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:57.862115 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] The token cache is being invalidate because project 795dd49fb5c84a85b8d318c41452f9dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:57.862401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:57.863020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:57.864493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1290/2569] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:57 2022] DELETE /identity/v3/projects/795dd49fb5c84a85b8d318c41452f9dc => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:57.868748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d775b-9e16-4a4d-b527-d12710943e2b service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:57.882939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d775b-9e16-4a4d-b527-d12710943e2b service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:57.892266 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:57.948996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30d065-c171-49bf-9062-13a432c3ab5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:57.949763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30d065-c171-49bf-9062-13a432c3ab5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:57.950082 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30d065-c171-49bf-9062-13a432c3ab5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:57.950355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30d065-c171-49bf-9062-13a432c3ab5a admin admin] PATH_INFO: `/v3/users/e6d44706e5404ed89ad1651d82034eab` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:57.951530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30d065-c171-49bf-9062-13a432c3ab5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6d44706e5404ed89ad1651d82034eab)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:57.953602 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30d065-c171-49bf-9062-13a432c3ab5a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:58.017113 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6f30d065-c171-49bf-9062-13a432c3ab5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6d44706e5404ed89ad1651d82034eab'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:58.027313 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6f30d065-c171-49bf-9062-13a432c3ab5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6d44706e5404ed89ad1651d82034eab'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:58.028013 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6f30d065-c171-49bf-9062-13a432c3ab5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6d44706e5404ed89ad1651d82034eab'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:58.028675 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6f30d065-c171-49bf-9062-13a432c3ab5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6d44706e5404ed89ad1651d82034eab'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:58.041079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-6f30d065-c171-49bf-9062-13a432c3ab5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6d44706e5404ed89ad1651d82034eab'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:58.067635 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1280/2570] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:00:57 2022] DELETE /identity/v3/users/e6d44706e5404ed89ad1651d82034eab => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:58.395752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:58.397639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:58.436494 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328691f-0511-4d0c-b8bb-bdeba94a59be admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:58.498594 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506615e9-2d88-4260-a5ff-cf8bdc264e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:58.499235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506615e9-2d88-4260-a5ff-cf8bdc264e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:58.499509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506615e9-2d88-4260-a5ff-cf8bdc264e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:58.499778 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506615e9-2d88-4260-a5ff-cf8bdc264e36 admin admin] PATH_INFO: `/v3/projects/ba275451ff584b079dd9c277472845e1` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:58.500926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506615e9-2d88-4260-a5ff-cf8bdc264e36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba275451ff584b079dd9c277472845e1)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:58.502846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506615e9-2d88-4260-a5ff-cf8bdc264e36 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:58.548838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-506615e9-2d88-4260-a5ff-cf8bdc264e36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba275451ff584b079dd9c277472845e1', 'request_id': 'req-506615e9-2d88-4260-a5ff-cf8bdc264e36'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:58.556403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-506615e9-2d88-4260-a5ff-cf8bdc264e36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba275451ff584b079dd9c277472845e1', 'request_id': 'req-506615e9-2d88-4260-a5ff-cf8bdc264e36'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:58.558941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:58.563948 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-506615e9-2d88-4260-a5ff-cf8bdc264e36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba275451ff584b079dd9c277472845e1', 'request_id': 'req-506615e9-2d88-4260-a5ff-cf8bdc264e36'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:58.575840 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-506615e9-2d88-4260-a5ff-cf8bdc264e36 admin admin] The token cache is being invalidate because project ba275451ff584b079dd9c277472845e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:00:58.576065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-506615e9-2d88-4260-a5ff-cf8bdc264e36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:58.576689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-506615e9-2d88-4260-a5ff-cf8bdc264e36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:00:58.578205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1291/2571] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:00:58 2022] DELETE /identity/v3/projects/ba275451ff584b079dd9c277472845e1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:58.583074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695476c3-c094-4252-a179-6acffc3a8732 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:58.583357 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695476c3-c094-4252-a179-6acffc3a8732 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:58.583628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695476c3-c094-4252-a179-6acffc3a8732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:58.602988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-695476c3-c094-4252-a179-6acffc3a8732 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:58.604170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-695476c3-c094-4252-a179-6acffc3a8732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:58.663648 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1281/2572] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:58.671413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d04ac-c484-43ab-82df-44f86b4a2905 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:58.671699 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d04ac-c484-43ab-82df-44f86b4a2905 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:58.671974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d04ac-c484-43ab-82df-44f86b4a2905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:58.692257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-e44d04ac-c484-43ab-82df-44f86b4a2905 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:58.693469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-e44d04ac-c484-43ab-82df-44f86b4a2905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:58.701907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1292/2573] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:00:58.706213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695476c3-c094-4252-a179-6acffc3a8732 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:58.716116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695476c3-c094-4252-a179-6acffc3a8732 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:58.720573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b45323-7fe7-49dc-b745-c6d1d9775ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:58.721163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b45323-7fe7-49dc-b745-c6d1d9775ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:58.721438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b45323-7fe7-49dc-b745-c6d1d9775ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:58.721702 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b45323-7fe7-49dc-b745-c6d1d9775ce4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:58.722363 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b45323-7fe7-49dc-b745-c6d1d9775ce4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:00:58.722659 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b45323-7fe7-49dc-b745-c6d1d9775ce4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:58.724272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b45323-7fe7-49dc-b745-c6d1d9775ce4 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:58.735426 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1282/2574] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:00:58.739599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d04ac-c484-43ab-82df-44f86b4a2905 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:58.750358 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d04ac-c484-43ab-82df-44f86b4a2905 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:58.755114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae7757-e6d0-458e-8452-e8516a4bb0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:58.755719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae7757-e6d0-458e-8452-e8516a4bb0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:58.755999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae7757-e6d0-458e-8452-e8516a4bb0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:58.756269 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae7757-e6d0-458e-8452-e8516a4bb0e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:58.756902 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae7757-e6d0-458e-8452-e8516a4bb0e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:58.758821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae7757-e6d0-458e-8452-e8516a4bb0e4 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:58.794744 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1293/2575] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:58 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:00:58.796727 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b45323-7fe7-49dc-b745-c6d1d9775ce4 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:58.808011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b45323-7fe7-49dc-b745-c6d1d9775ce4 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:58.824135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:58.872481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427134d-857d-4301-9f3a-9b61ca8ebdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:58.873195 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427134d-857d-4301-9f3a-9b61ca8ebdcf admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:58.873354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427134d-857d-4301-9f3a-9b61ca8ebdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:58.873624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427134d-857d-4301-9f3a-9b61ca8ebdcf admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:58.874299 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427134d-857d-4301-9f3a-9b61ca8ebdcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:58.876192 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427134d-857d-4301-9f3a-9b61ca8ebdcf admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:58.905406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1283/2576] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:58 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:58.909534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae7757-e6d0-458e-8452-e8516a4bb0e4 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:58.920730 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae7757-e6d0-458e-8452-e8516a4bb0e4 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:58.925639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e48460-76d0-4506-a004-3b9f7c9c7108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:58.926256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e48460-76d0-4506-a004-3b9f7c9c7108 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:58.926546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e48460-76d0-4506-a004-3b9f7c9c7108 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:58.926797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e48460-76d0-4506-a004-3b9f7c9c7108 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:58.927465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e48460-76d0-4506-a004-3b9f7c9c7108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:58.929090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e48460-76d0-4506-a004-3b9f7c9c7108 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:58.940217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1294/2577] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:58.944338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427134d-857d-4301-9f3a-9b61ca8ebdcf admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:58.955164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427134d-857d-4301-9f3a-9b61ca8ebdcf admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:58.959961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b8df3-e947-44df-874d-6f81d1c0ffe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:58.960562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b8df3-e947-44df-874d-6f81d1c0ffe7 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:58.960842 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b8df3-e947-44df-874d-6f81d1c0ffe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:58.961169 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b8df3-e947-44df-874d-6f81d1c0ffe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:58.961854 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b8df3-e947-44df-874d-6f81d1c0ffe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:58.963505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b8df3-e947-44df-874d-6f81d1c0ffe7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:58.975094 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1284/2578] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:00:58.980033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e48460-76d0-4506-a004-3b9f7c9c7108 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:58.990090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e48460-76d0-4506-a004-3b9f7c9c7108 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:58.995517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4223adb-2851-4b5e-a9fd-9fc5555d0659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:58.996150 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4223adb-2851-4b5e-a9fd-9fc5555d0659 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:58.996424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4223adb-2851-4b5e-a9fd-9fc5555d0659 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:58.996696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4223adb-2851-4b5e-a9fd-9fc5555d0659 admin admin] PATH_INFO: `/v3/projects/c5e8ff71a0d34ef0b06ee65fdd527ff4/users/f404f860d0394059b3108796d268538a/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:59.011250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4223adb-2851-4b5e-a9fd-9fc5555d0659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e8ff71a0d34ef0b06ee65fdd527ff4, user_id=f404f860d0394059b3108796d268538a, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:59.013979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4223adb-2851-4b5e-a9fd-9fc5555d0659 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:59.026546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1295/2579] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:58 2022] PUT /identity/v3/projects/c5e8ff71a0d34ef0b06ee65fdd527ff4/users/f404f860d0394059b3108796d268538a/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:59.036953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9017e1-3a71-40ad-8a70-ad61d7f3e0b8 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:59.037238 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9017e1-3a71-40ad-8a70-ad61d7f3e0b8 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:59.037538 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9017e1-3a71-40ad-8a70-ad61d7f3e0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:59.066623 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6d9017e1-3a71-40ad-8a70-ad61d7f3e0b8 None None] MFA Rules not processed for user `f404f860d0394059b3108796d268538a`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:59.067828 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6d9017e1-3a71-40ad-8a70-ad61d7f3e0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:59.142581 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1285/2580] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:59.146736 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4223adb-2851-4b5e-a9fd-9fc5555d0659 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:59.210998 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4223adb-2851-4b5e-a9fd-9fc5555d0659 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:59.216505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5dda8f-66dd-46c5-b829-bf87d08bb80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:59.217106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dda8f-66dd-46c5-b829-bf87d08bb80f admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:59.217417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dda8f-66dd-46c5-b829-bf87d08bb80f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:59.217688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dda8f-66dd-46c5-b829-bf87d08bb80f admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:59.218377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5dda8f-66dd-46c5-b829-bf87d08bb80f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:59.220231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5dda8f-66dd-46c5-b829-bf87d08bb80f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:59.249512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1296/2581] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:00:59 2022] POST /identity/v3/projects => generated 369 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:00:59.253543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9017e1-3a71-40ad-8a70-ad61d7f3e0b8 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:59.265337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9017e1-3a71-40ad-8a70-ad61d7f3e0b8 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:59.327040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700d1f4-49e7-4e6a-a922-1308b6898c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:59.327658 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700d1f4-49e7-4e6a-a922-1308b6898c48 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:59.327937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700d1f4-49e7-4e6a-a922-1308b6898c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:59.328213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700d1f4-49e7-4e6a-a922-1308b6898c48 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:59.328862 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700d1f4-49e7-4e6a-a922-1308b6898c48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:59.330944 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700d1f4-49e7-4e6a-a922-1308b6898c48 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:59.352395 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1286/2582] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:00:59 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:00:59.356497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5dda8f-66dd-46c5-b829-bf87d08bb80f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:59.366802 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5dda8f-66dd-46c5-b829-bf87d08bb80f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:59.372135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384fd6b-f4a9-448b-ae68-670f3e21616d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:59.372730 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384fd6b-f4a9-448b-ae68-670f3e21616d admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:59.373008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384fd6b-f4a9-448b-ae68-670f3e21616d admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:59.373286 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384fd6b-f4a9-448b-ae68-670f3e21616d admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:59.373980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384fd6b-f4a9-448b-ae68-670f3e21616d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:59.375551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384fd6b-f4a9-448b-ae68-670f3e21616d admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:59.393141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1297/2583] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:59.397454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700d1f4-49e7-4e6a-a922-1308b6898c48 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:59.408628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700d1f4-49e7-4e6a-a922-1308b6898c48 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:59.413535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8b6f2-4af6-4c81-99f4-633862179fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:59.414239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8b6f2-4af6-4c81-99f4-633862179fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:59.414520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8b6f2-4af6-4c81-99f4-633862179fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:59.414837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc8b6f2-4af6-4c81-99f4-633862179fc7 admin admin] PATH_INFO: `/v3/projects/b1ad331e15904498b00f402898f67532/users/2b74c50e3d954cacbf623348f8aa01fd/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:59.428472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:59.432770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8b6f2-4af6-4c81-99f4-633862179fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ad331e15904498b00f402898f67532, user_id=2b74c50e3d954cacbf623348f8aa01fd, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:59.435646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc8b6f2-4af6-4c81-99f4-633862179fc7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:59.453843 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1287/2584] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:59 2022] PUT /identity/v3/projects/b1ad331e15904498b00f402898f67532/users/2b74c50e3d954cacbf623348f8aa01fd/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:59.455040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384fd6b-f4a9-448b-ae68-670f3e21616d admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:59.467125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384fd6b-f4a9-448b-ae68-670f3e21616d admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:59.526646 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:59.536687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96224030-1b5b-4287-bee2-828f56eed2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:59.537785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96224030-1b5b-4287-bee2-828f56eed2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:59.538261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96224030-1b5b-4287-bee2-828f56eed2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:59.538922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96224030-1b5b-4287-bee2-828f56eed2e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:59.539829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96224030-1b5b-4287-bee2-828f56eed2e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:59.541355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96224030-1b5b-4287-bee2-828f56eed2e8 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:59.557138 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1298/2585] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:00:59.558513 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8b6f2-4af6-4c81-99f4-633862179fc7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:59.578172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc8b6f2-4af6-4c81-99f4-633862179fc7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:59.583973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60897da-e683-442b-bba8-39a622fdcc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:59.584821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60897da-e683-442b-bba8-39a622fdcc2a admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:59.585099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60897da-e683-442b-bba8-39a622fdcc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:59.585397 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60897da-e683-442b-bba8-39a622fdcc2a admin admin] PATH_INFO: `/v3/projects/b1ad331e15904498b00f402898f67532/users/2b74c50e3d954cacbf623348f8aa01fd/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:59.588599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60897da-e683-442b-bba8-39a622fdcc2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ad331e15904498b00f402898f67532, user_id=2b74c50e3d954cacbf623348f8aa01fd, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:59.591823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60897da-e683-442b-bba8-39a622fdcc2a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:59.605547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1288/2586] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:00:59 2022] PUT /identity/v3/projects/b1ad331e15904498b00f402898f67532/users/2b74c50e3d954cacbf623348f8aa01fd/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:00:59.617257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83d2a-0b5a-45b5-9def-685146873ffd None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:59.617542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83d2a-0b5a-45b5-9def-685146873ffd None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:59.617863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83d2a-0b5a-45b5-9def-685146873ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:59.648939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-5ac83d2a-0b5a-45b5-9def-685146873ffd None None] MFA Rules not processed for user `2b74c50e3d954cacbf623348f8aa01fd`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:00:59.650385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5ac83d2a-0b5a-45b5-9def-685146873ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:59.746035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1299/2587] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:00:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:00:59.757303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60897da-e683-442b-bba8-39a622fdcc2a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:59.764815 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b60897da-e683-442b-bba8-39a622fdcc2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:59.832344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60897da-e683-442b-bba8-39a622fdcc2a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:59.836310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55997b0-6e49-4a75-bb92-700acbe574d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:59.836861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55997b0-6e49-4a75-bb92-700acbe574d1 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:59.837633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55997b0-6e49-4a75-bb92-700acbe574d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:59.837633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55997b0-6e49-4a75-bb92-700acbe574d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:59.847427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55997b0-6e49-4a75-bb92-700acbe574d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:59.849107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55997b0-6e49-4a75-bb92-700acbe574d1 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:00:59.860629 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1289/2588] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:00:59.914963 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac83d2a-0b5a-45b5-9def-685146873ffd None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:00:59.917048 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5ac83d2a-0b5a-45b5-9def-685146873ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:00:59.931970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac83d2a-0b5a-45b5-9def-685146873ffd None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:00:59.935936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7564c-537f-4a0b-ba57-11e20b3770cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:00:59.936546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7564c-537f-4a0b-ba57-11e20b3770cd service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:00:59.936925 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7564c-537f-4a0b-ba57-11e20b3770cd service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:00:59.937208 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7564c-537f-4a0b-ba57-11e20b3770cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:00:59.996350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7564c-537f-4a0b-ba57-11e20b3770cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:00:59.997841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7564c-537f-4a0b-ba57-11e20b3770cd service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:00.007185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1300/2589] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:00:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:00.210878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173508eb-4210-4bbd-b867-35623eaacce1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:00.211381 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173508eb-4210-4bbd-b867-35623eaacce1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:00.211518 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173508eb-4210-4bbd-b867-35623eaacce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:00.234290 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-173508eb-4210-4bbd-b867-35623eaacce1 None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:00.235601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-173508eb-4210-4bbd-b867-35623eaacce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:00.306239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1290/2590] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 14:01:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:00.319183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7564c-537f-4a0b-ba57-11e20b3770cd service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:00.321177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-44c7564c-537f-4a0b-ba57-11e20b3770cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:00.392530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7564c-537f-4a0b-ba57-11e20b3770cd service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:00.397512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c1af9-ad3a-4295-832d-a5215e0588c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:00.398140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c1af9-ad3a-4295-832d-a5215e0588c7 service swift] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:00.398422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c1af9-ad3a-4295-832d-a5215e0588c7 service swift] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:00.398695 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c1af9-ad3a-4295-832d-a5215e0588c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:00.407220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c1af9-ad3a-4295-832d-a5215e0588c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:00.408816 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c1af9-ad3a-4295-832d-a5215e0588c7 service swift] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:00.416826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1301/2591] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 14:01:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:01:03.644062 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173508eb-4210-4bbd-b867-35623eaacce1 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:03.794489 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173508eb-4210-4bbd-b867-35623eaacce1 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:03.818637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:03.819695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:03.819983 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:03.820263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] PATH_INFO: `/v3/users/f404f860d0394059b3108796d268538a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:03.821515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f404f860d0394059b3108796d268538a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:03.825292 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:03.920502 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:03.945571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f404f860d0394059b3108796d268538a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:03.966214 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f404f860d0394059b3108796d268538a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:03.967712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f404f860d0394059b3108796d268538a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:03.969240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f404f860d0394059b3108796d268538a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:03.984109 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f404f860d0394059b3108796d268538a'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:04.008623 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1291/2592] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:03 2022] DELETE /identity/v3/users/f404f860d0394059b3108796d268538a => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:04.010352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef1c46-a5ac-40cc-97d7-8b345a3a16ef None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:04.010838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef1c46-a5ac-40cc-97d7-8b345a3a16ef None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:04.010933 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef1c46-a5ac-40cc-97d7-8b345a3a16ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:04.023084 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:04.037696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4eef1c46-a5ac-40cc-97d7-8b345a3a16ef None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:04.039030 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4eef1c46-a5ac-40cc-97d7-8b345a3a16ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:04.151992 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1302/2593] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:04.161975 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:04.163672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:04.294388 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9c1bc-9fbd-4e47-a303-8b697879ea49 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:04.304730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628965c-6abf-4f12-88b1-57cbc7585a61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:04.305482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628965c-6abf-4f12-88b1-57cbc7585a61 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:04.306387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628965c-6abf-4f12-88b1-57cbc7585a61 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:04.306868 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628965c-6abf-4f12-88b1-57cbc7585a61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:04.317312 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628965c-6abf-4f12-88b1-57cbc7585a61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:04.318921 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628965c-6abf-4f12-88b1-57cbc7585a61 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:04.330992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1292/2594] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:04.655300 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef1c46-a5ac-40cc-97d7-8b345a3a16ef None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:04.656949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4eef1c46-a5ac-40cc-97d7-8b345a3a16ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:04.675812 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef1c46-a5ac-40cc-97d7-8b345a3a16ef None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:04.682210 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:04.682848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:04.683126 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:04.683397 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] PATH_INFO: `/v3/projects/c5e8ff71a0d34ef0b06ee65fdd527ff4` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:04.684536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e8ff71a0d34ef0b06ee65fdd527ff4)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:04.686572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:04.769947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e8ff71a0d34ef0b06ee65fdd527ff4', 'request_id': 'req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:04.777012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e8ff71a0d34ef0b06ee65fdd527ff4', 'request_id': 'req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:04.785998 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e8ff71a0d34ef0b06ee65fdd527ff4', 'request_id': 'req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:04.798021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] The token cache is being invalidate because project c5e8ff71a0d34ef0b06ee65fdd527ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:04.798525 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:04.799219 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:04.801313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1303/2595] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:04 2022] DELETE /identity/v3/projects/c5e8ff71a0d34ef0b06ee65fdd527ff4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:04.805092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628965c-6abf-4f12-88b1-57cbc7585a61 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:04.815923 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628965c-6abf-4f12-88b1-57cbc7585a61 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:04.933571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74e502-eb62-4f87-ba5f-f1591c3c67b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:04.933571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74e502-eb62-4f87-ba5f-f1591c3c67b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:04.933571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74e502-eb62-4f87-ba5f-f1591c3c67b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:04.934053 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74e502-eb62-4f87-ba5f-f1591c3c67b2 admin admin] PATH_INFO: `/v3/users/2b74c50e3d954cacbf623348f8aa01fd` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:04.935336 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74e502-eb62-4f87-ba5f-f1591c3c67b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b74c50e3d954cacbf623348f8aa01fd)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:04.937843 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74e502-eb62-4f87-ba5f-f1591c3c67b2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:04.997432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f74e502-eb62-4f87-ba5f-f1591c3c67b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b74c50e3d954cacbf623348f8aa01fd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.001152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:05.011501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f74e502-eb62-4f87-ba5f-f1591c3c67b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b74c50e3d954cacbf623348f8aa01fd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.012255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f74e502-eb62-4f87-ba5f-f1591c3c67b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b74c50e3d954cacbf623348f8aa01fd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.014084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f74e502-eb62-4f87-ba5f-f1591c3c67b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b74c50e3d954cacbf623348f8aa01fd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.028690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9f74e502-eb62-4f87-ba5f-f1591c3c67b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b74c50e3d954cacbf623348f8aa01fd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.059300 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1293/2596] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:04 2022] DELETE /identity/v3/users/2b74c50e3d954cacbf623348f8aa01fd => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:05.465350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:05.467543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:05.512318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d84f0-29aa-4419-bcb5-ccc1f7e8517f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:05.665604 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00411ea-2049-42d5-96fe-865ce2dd3c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:05.666486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00411ea-2049-42d5-96fe-865ce2dd3c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:05.666764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00411ea-2049-42d5-96fe-865ce2dd3c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:05.667033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00411ea-2049-42d5-96fe-865ce2dd3c7e admin admin] PATH_INFO: `/v3/projects/b1ad331e15904498b00f402898f67532` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:05.668615 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00411ea-2049-42d5-96fe-865ce2dd3c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1ad331e15904498b00f402898f67532)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:05.670910 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00411ea-2049-42d5-96fe-865ce2dd3c7e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:05.724140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:05.783069 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a00411ea-2049-42d5-96fe-865ce2dd3c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1ad331e15904498b00f402898f67532', 'request_id': 'req-a00411ea-2049-42d5-96fe-865ce2dd3c7e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.798690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a00411ea-2049-42d5-96fe-865ce2dd3c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1ad331e15904498b00f402898f67532', 'request_id': 'req-a00411ea-2049-42d5-96fe-865ce2dd3c7e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.818968 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a00411ea-2049-42d5-96fe-865ce2dd3c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1ad331e15904498b00f402898f67532', 'request_id': 'req-a00411ea-2049-42d5-96fe-865ce2dd3c7e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.845076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a00411ea-2049-42d5-96fe-865ce2dd3c7e admin admin] The token cache is being invalidate because project b1ad331e15904498b00f402898f67532 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:05.845977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a00411ea-2049-42d5-96fe-865ce2dd3c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.846944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a00411ea-2049-42d5-96fe-865ce2dd3c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:05.848984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1304/2597] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:05 2022] DELETE /identity/v3/projects/b1ad331e15904498b00f402898f67532 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:05.854371 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbffc0-812e-47c6-ac0b-f0a3a7dc9b18 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:05.854953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbffc0-812e-47c6-ac0b-f0a3a7dc9b18 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:05.854953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbffc0-812e-47c6-ac0b-f0a3a7dc9b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:05.884424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-72bbffc0-812e-47c6-ac0b-f0a3a7dc9b18 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:05.885992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-72bbffc0-812e-47c6-ac0b-f0a3a7dc9b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:05.986533 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1294/2598] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:05.995149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b2e99-1ee5-4fde-b7c3-1aa047c4e71f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:05.995437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b2e99-1ee5-4fde-b7c3-1aa047c4e71f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:05.995690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b2e99-1ee5-4fde-b7c3-1aa047c4e71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.019673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4e2b2e99-1ee5-4fde-b7c3-1aa047c4e71f None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:06.020920 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4e2b2e99-1ee5-4fde-b7c3-1aa047c4e71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:06.029008 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1305/2599] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:06.033530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbffc0-812e-47c6-ac0b-f0a3a7dc9b18 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.047426 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bbffc0-812e-47c6-ac0b-f0a3a7dc9b18 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.052157 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7df75-31c6-4d2c-b00c-74b355abe3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.052776 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7df75-31c6-4d2c-b00c-74b355abe3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.053058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7df75-31c6-4d2c-b00c-74b355abe3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.053329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7df75-31c6-4d2c-b00c-74b355abe3d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.054005 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7df75-31c6-4d2c-b00c-74b355abe3d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:06.054311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7df75-31c6-4d2c-b00c-74b355abe3d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.056540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7df75-31c6-4d2c-b00c-74b355abe3d1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.077782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1295/2600] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:01:06.079455 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b2e99-1ee5-4fde-b7c3-1aa047c4e71f None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.096448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b2e99-1ee5-4fde-b7c3-1aa047c4e71f None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.101333 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e0823-1281-42eb-b1f2-8f895b35ba7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.102110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e0823-1281-42eb-b1f2-8f895b35ba7b admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.102390 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e0823-1281-42eb-b1f2-8f895b35ba7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.102824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e0823-1281-42eb-b1f2-8f895b35ba7b admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.103542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e0823-1281-42eb-b1f2-8f895b35ba7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.105815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e0823-1281-42eb-b1f2-8f895b35ba7b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.155380 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1306/2601] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:06 2022] POST /identity/v3/projects => generated 367 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:06.159616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7df75-31c6-4d2c-b00c-74b355abe3d1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.170001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7df75-31c6-4d2c-b00c-74b355abe3d1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.267167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e446e0-e1e7-4c9d-aeff-03b6dea99910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.268653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e446e0-e1e7-4c9d-aeff-03b6dea99910 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.269096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e446e0-e1e7-4c9d-aeff-03b6dea99910 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.269570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e446e0-e1e7-4c9d-aeff-03b6dea99910 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.270429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e446e0-e1e7-4c9d-aeff-03b6dea99910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.272825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e446e0-e1e7-4c9d-aeff-03b6dea99910 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.304398 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1296/2602] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:06 2022] POST /identity/v3/users => generated 284 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:06.306328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e0823-1281-42eb-b1f2-8f895b35ba7b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.318388 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e0823-1281-42eb-b1f2-8f895b35ba7b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.325001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfae19-635a-4d5c-b3d0-24a1c1097cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.325621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfae19-635a-4d5c-b3d0-24a1c1097cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.325936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfae19-635a-4d5c-b3d0-24a1c1097cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.326194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfae19-635a-4d5c-b3d0-24a1c1097cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.326983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfae19-635a-4d5c-b3d0-24a1c1097cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.329273 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bfae19-635a-4d5c-b3d0-24a1c1097cd5 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.344724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1307/2603] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:06.348221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e446e0-e1e7-4c9d-aeff-03b6dea99910 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.360539 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e446e0-e1e7-4c9d-aeff-03b6dea99910 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.366612 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fd540-ccc7-43ec-baaf-bca6fe2c19ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.367406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fd540-ccc7-43ec-baaf-bca6fe2c19ce admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.367986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fd540-ccc7-43ec-baaf-bca6fe2c19ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.369056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fd540-ccc7-43ec-baaf-bca6fe2c19ce admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.370014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fd540-ccc7-43ec-baaf-bca6fe2c19ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.372667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fd540-ccc7-43ec-baaf-bca6fe2c19ce admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.389898 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1297/2604] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:06.393818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfae19-635a-4d5c-b3d0-24a1c1097cd5 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.405631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfae19-635a-4d5c-b3d0-24a1c1097cd5 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.412746 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f8397-c65e-4c67-b35e-4927c704bb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.413697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f8397-c65e-4c67-b35e-4927c704bb9f admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.414104 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f8397-c65e-4c67-b35e-4927c704bb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.414476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76f8397-c65e-4c67-b35e-4927c704bb9f admin admin] PATH_INFO: `/v3/projects/8ea0aa6b367c4b97a99b64b16f4b81ae/users/9c614c370d5946ffae63536d51ad891d/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.458633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f8397-c65e-4c67-b35e-4927c704bb9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea0aa6b367c4b97a99b64b16f4b81ae, user_id=9c614c370d5946ffae63536d51ad891d, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.461287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76f8397-c65e-4c67-b35e-4927c704bb9f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.490721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1308/2605] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:06 2022] PUT /identity/v3/projects/8ea0aa6b367c4b97a99b64b16f4b81ae/users/9c614c370d5946ffae63536d51ad891d/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:06.501943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce996b-ec3d-4db3-a2d7-fe9b8c679388 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.502320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce996b-ec3d-4db3-a2d7-fe9b8c679388 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.502664 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce996b-ec3d-4db3-a2d7-fe9b8c679388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.536917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-abce996b-ec3d-4db3-a2d7-fe9b8c679388 None None] MFA Rules not processed for user `9c614c370d5946ffae63536d51ad891d`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:06.538227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-abce996b-ec3d-4db3-a2d7-fe9b8c679388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:06.562694 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:06.623544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1298/2606] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:06.631428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f8397-c65e-4c67-b35e-4927c704bb9f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.697086 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76f8397-c65e-4c67-b35e-4927c704bb9f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.702230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d49dd5-8402-4f13-a566-b5eede24268b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.702955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d49dd5-8402-4f13-a566-b5eede24268b admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.703278 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d49dd5-8402-4f13-a566-b5eede24268b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.703621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d49dd5-8402-4f13-a566-b5eede24268b admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.704329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d49dd5-8402-4f13-a566-b5eede24268b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.706362 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d49dd5-8402-4f13-a566-b5eede24268b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.737947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1309/2607] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:06 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:06.742301 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce996b-ec3d-4db3-a2d7-fe9b8c679388 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.769817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce996b-ec3d-4db3-a2d7-fe9b8c679388 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.839418 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b713c-391d-49d3-b8e7-1190bc7a1c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.840515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b713c-391d-49d3-b8e7-1190bc7a1c98 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.840842 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b713c-391d-49d3-b8e7-1190bc7a1c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.841122 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b713c-391d-49d3-b8e7-1190bc7a1c98 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.841847 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b713c-391d-49d3-b8e7-1190bc7a1c98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.843913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b713c-391d-49d3-b8e7-1190bc7a1c98 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.871250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1299/2608] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:06 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:06.875021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d49dd5-8402-4f13-a566-b5eede24268b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.888824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d49dd5-8402-4f13-a566-b5eede24268b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.893953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9d2c3-ae5d-4d3e-8ead-6846f06fe41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.894570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9d2c3-ae5d-4d3e-8ead-6846f06fe41a admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.894847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9d2c3-ae5d-4d3e-8ead-6846f06fe41a admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.895116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9d2c3-ae5d-4d3e-8ead-6846f06fe41a admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.895773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9d2c3-ae5d-4d3e-8ead-6846f06fe41a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.897376 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9d2c3-ae5d-4d3e-8ead-6846f06fe41a admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:06.914656 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1310/2609] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:06.918988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b713c-391d-49d3-b8e7-1190bc7a1c98 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:06.935814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b713c-391d-49d3-b8e7-1190bc7a1c98 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:06.944832 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7779e0-78b5-4d45-ac14-9a4107cc9766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:06.945855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7779e0-78b5-4d45-ac14-9a4107cc9766 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:06.946530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7779e0-78b5-4d45-ac14-9a4107cc9766 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:06.946973 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7779e0-78b5-4d45-ac14-9a4107cc9766 admin admin] PATH_INFO: `/v3/projects/c1c6a9a177554694b998a0531d07ffb4/users/1108eab174bf4f6a92e975fefa015c15/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:06.985499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7779e0-78b5-4d45-ac14-9a4107cc9766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c6a9a177554694b998a0531d07ffb4, user_id=1108eab174bf4f6a92e975fefa015c15, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:06.991125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7779e0-78b5-4d45-ac14-9a4107cc9766 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.016589 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1300/2610] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:06 2022] PUT /identity/v3/projects/c1c6a9a177554694b998a0531d07ffb4/users/1108eab174bf4f6a92e975fefa015c15/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:07.021099 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9d2c3-ae5d-4d3e-8ead-6846f06fe41a admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.035977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9d2c3-ae5d-4d3e-8ead-6846f06fe41a admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:07.069725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:07.117267 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a5897a-0224-4c4a-ab96-fcaa8cc60090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:07.117970 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5897a-0224-4c4a-ab96-fcaa8cc60090 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.118289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5897a-0224-4c4a-ab96-fcaa8cc60090 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.118568 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a5897a-0224-4c4a-ab96-fcaa8cc60090 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.119217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5897a-0224-4c4a-ab96-fcaa8cc60090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:07.120896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a5897a-0224-4c4a-ab96-fcaa8cc60090 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.132293 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1311/2611] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:07.137241 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7779e0-78b5-4d45-ac14-9a4107cc9766 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.150234 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7779e0-78b5-4d45-ac14-9a4107cc9766 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:07.175488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ea3ce-7d32-47d0-bde2-9242e18dbda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:07.176527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ea3ce-7d32-47d0-bde2-9242e18dbda7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.177095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ea3ce-7d32-47d0-bde2-9242e18dbda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.177986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ea3ce-7d32-47d0-bde2-9242e18dbda7 admin admin] PATH_INFO: `/v3/projects/c1c6a9a177554694b998a0531d07ffb4/users/1108eab174bf4f6a92e975fefa015c15/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.183924 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ea3ce-7d32-47d0-bde2-9242e18dbda7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c6a9a177554694b998a0531d07ffb4, user_id=1108eab174bf4f6a92e975fefa015c15, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:07.189610 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ea3ce-7d32-47d0-bde2-9242e18dbda7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.230420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1301/2612] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:07 2022] PUT /identity/v3/projects/c1c6a9a177554694b998a0531d07ffb4/users/1108eab174bf4f6a92e975fefa015c15/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:07.246381 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf09165-03fe-430e-b2b9-1ff7c8eff9a1 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.246910 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf09165-03fe-430e-b2b9-1ff7c8eff9a1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.247231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf09165-03fe-430e-b2b9-1ff7c8eff9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.320045 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-5bf09165-03fe-430e-b2b9-1ff7c8eff9a1 None None] MFA Rules not processed for user `1108eab174bf4f6a92e975fefa015c15`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:07.321507 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5bf09165-03fe-430e-b2b9-1ff7c8eff9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:07.549961 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1312/2613] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:07.559752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ea3ce-7d32-47d0-bde2-9242e18dbda7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.561989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-513ea3ce-7d32-47d0-bde2-9242e18dbda7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:07.663251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ea3ce-7d32-47d0-bde2-9242e18dbda7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:07.670788 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0412ef-8ea6-47da-ac5b-b54dba441356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:07.671641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0412ef-8ea6-47da-ac5b-b54dba441356 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.672078 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0412ef-8ea6-47da-ac5b-b54dba441356 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.672418 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0412ef-8ea6-47da-ac5b-b54dba441356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.683573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0412ef-8ea6-47da-ac5b-b54dba441356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:07.685401 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0412ef-8ea6-47da-ac5b-b54dba441356 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.709945 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1302/2614] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:07.794489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf09165-03fe-430e-b2b9-1ff7c8eff9a1 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:07.804147 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf09165-03fe-430e-b2b9-1ff7c8eff9a1 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:07.807748 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657278ba-44bc-45ec-9538-d1917203f1b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:07.808344 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657278ba-44bc-45ec-9538-d1917203f1b4 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:07.808624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657278ba-44bc-45ec-9538-d1917203f1b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:07.808896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657278ba-44bc-45ec-9538-d1917203f1b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:07.866224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657278ba-44bc-45ec-9538-d1917203f1b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:07.867650 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657278ba-44bc-45ec-9538-d1917203f1b4 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:07.876669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1313/2615] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:12.648335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0412ef-8ea6-47da-ac5b-b54dba441356 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:12.669403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:12.734509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0412ef-8ea6-47da-ac5b-b54dba441356 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:12.743814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:12.744492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:12.744772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:12.745045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] PATH_INFO: `/v3/users/9c614c370d5946ffae63536d51ad891d` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:12.746242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c614c370d5946ffae63536d51ad891d)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:12.748217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:12.870610 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c614c370d5946ffae63536d51ad891d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:12.878663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c614c370d5946ffae63536d51ad891d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:12.879406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c614c370d5946ffae63536d51ad891d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:12.880070 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c614c370d5946ffae63536d51ad891d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:12.893760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c614c370d5946ffae63536d51ad891d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:12.910029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1303/2616] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:12 2022] DELETE /identity/v3/users/9c614c370d5946ffae63536d51ad891d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:12.914253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c1d1c-6b16-49ac-b3d8-ad10365e7d0c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:12.914675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c1d1c-6b16-49ac-b3d8-ad10365e7d0c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:12.914901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c1d1c-6b16-49ac-b3d8-ad10365e7d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:12.934686 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-0e2c1d1c-6b16-49ac-b3d8-ad10365e7d0c None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:12.935972 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0e2c1d1c-6b16-49ac-b3d8-ad10365e7d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:12.985712 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:13.001410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1314/2617] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:13.011260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:13.012764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:13.078787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a67ae-1999-48fb-b0e1-8a5d799565c2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:13.082393 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90159e-7d38-428a-9be5-a42bc37f3f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:13.082995 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90159e-7d38-428a-9be5-a42bc37f3f5e service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:13.083272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90159e-7d38-428a-9be5-a42bc37f3f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:13.083598 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90159e-7d38-428a-9be5-a42bc37f3f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:13.093261 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90159e-7d38-428a-9be5-a42bc37f3f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:13.094706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90159e-7d38-428a-9be5-a42bc37f3f5e service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:13.104495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1304/2618] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:13.398301 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c1d1c-6b16-49ac-b3d8-ad10365e7d0c None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:13.400458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0e2c1d1c-6b16-49ac-b3d8-ad10365e7d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:13.424497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c1d1c-6b16-49ac-b3d8-ad10365e7d0c None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:13.429577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:13.430402 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:13.430757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:13.431031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] PATH_INFO: `/v3/projects/8ea0aa6b367c4b97a99b64b16f4b81ae` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:13.432203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ea0aa6b367c4b97a99b64b16f4b81ae)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:13.434205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:13.534746 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ea0aa6b367c4b97a99b64b16f4b81ae', 'request_id': 'req-d0ebec51-6caf-4a59-ad48-ceaa6131482e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:13.543832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ea0aa6b367c4b97a99b64b16f4b81ae', 'request_id': 'req-d0ebec51-6caf-4a59-ad48-ceaa6131482e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:13.561518 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ea0aa6b367c4b97a99b64b16f4b81ae', 'request_id': 'req-d0ebec51-6caf-4a59-ad48-ceaa6131482e'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:13.581100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] The token cache is being invalidate because project 8ea0aa6b367c4b97a99b64b16f4b81ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:13.581662 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:13.582659 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:13.584491 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1315/2619] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:13 2022] DELETE /identity/v3/projects/8ea0aa6b367c4b97a99b64b16f4b81ae => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:13.591114 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90159e-7d38-428a-9be5-a42bc37f3f5e service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:13.603012 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90159e-7d38-428a-9be5-a42bc37f3f5e service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:13.663131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:13.691046 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366938d1-af09-497f-9ec2-805a7208ae80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:13.698610 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366938d1-af09-497f-9ec2-805a7208ae80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:13.699243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366938d1-af09-497f-9ec2-805a7208ae80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:13.699588 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366938d1-af09-497f-9ec2-805a7208ae80 admin admin] PATH_INFO: `/v3/users/1108eab174bf4f6a92e975fefa015c15` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:13.700998 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366938d1-af09-497f-9ec2-805a7208ae80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1108eab174bf4f6a92e975fefa015c15)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:13.703120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366938d1-af09-497f-9ec2-805a7208ae80 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:13.764627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-366938d1-af09-497f-9ec2-805a7208ae80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1108eab174bf4f6a92e975fefa015c15'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:13.777207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-366938d1-af09-497f-9ec2-805a7208ae80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1108eab174bf4f6a92e975fefa015c15'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:13.778134 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-366938d1-af09-497f-9ec2-805a7208ae80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1108eab174bf4f6a92e975fefa015c15'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:13.778996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-366938d1-af09-497f-9ec2-805a7208ae80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1108eab174bf4f6a92e975fefa015c15'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:13.791957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-366938d1-af09-497f-9ec2-805a7208ae80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1108eab174bf4f6a92e975fefa015c15'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:13.818923 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1305/2620] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:13 2022] DELETE /identity/v3/users/1108eab174bf4f6a92e975fefa015c15 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:14.156241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:14.159048 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:14.178793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebec51-6caf-4a59-ad48-ceaa6131482e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:14.253322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcc76d-2437-4a84-98da-5001e0bb8575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:14.254050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc76d-2437-4a84-98da-5001e0bb8575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:14.254343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc76d-2437-4a84-98da-5001e0bb8575 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:14.254603 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc76d-2437-4a84-98da-5001e0bb8575 admin admin] PATH_INFO: `/v3/projects/c1c6a9a177554694b998a0531d07ffb4` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:14.255926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcc76d-2437-4a84-98da-5001e0bb8575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c6a9a177554694b998a0531d07ffb4)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:14.258048 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcc76d-2437-4a84-98da-5001e0bb8575 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:14.316516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ecfcc76d-2437-4a84-98da-5001e0bb8575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c6a9a177554694b998a0531d07ffb4', 'request_id': 'req-ecfcc76d-2437-4a84-98da-5001e0bb8575'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:14.324614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ecfcc76d-2437-4a84-98da-5001e0bb8575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c6a9a177554694b998a0531d07ffb4', 'request_id': 'req-ecfcc76d-2437-4a84-98da-5001e0bb8575'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:14.335413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ecfcc76d-2437-4a84-98da-5001e0bb8575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c6a9a177554694b998a0531d07ffb4', 'request_id': 'req-ecfcc76d-2437-4a84-98da-5001e0bb8575'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:14.350291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ecfcc76d-2437-4a84-98da-5001e0bb8575 admin admin] The token cache is being invalidate because project c1c6a9a177554694b998a0531d07ffb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:14.350720 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ecfcc76d-2437-4a84-98da-5001e0bb8575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:14.351510 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-ecfcc76d-2437-4a84-98da-5001e0bb8575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:14.353507 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1316/2621] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:14 2022] DELETE /identity/v3/projects/c1c6a9a177554694b998a0531d07ffb4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:14.358550 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bdd1d-ce24-47ea-abff-e5b7a011f6e5 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:14.359092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bdd1d-ce24-47ea-abff-e5b7a011f6e5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:14.359423 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bdd1d-ce24-47ea-abff-e5b7a011f6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:14.383720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2f0bdd1d-ce24-47ea-abff-e5b7a011f6e5 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:14.385359 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2f0bdd1d-ce24-47ea-abff-e5b7a011f6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:14.451968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1306/2622] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:14.460083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dce673-7f84-4034-8116-d083b386dad2 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:14.460438 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dce673-7f84-4034-8116-d083b386dad2 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:14.460780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dce673-7f84-4034-8116-d083b386dad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:14.469706 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.486745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-84dce673-7f84-4034-8116-d083b386dad2 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:14.487956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-84dce673-7f84-4034-8116-d083b386dad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:14.495854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1317/2623] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:14.500115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bdd1d-ce24-47ea-abff-e5b7a011f6e5 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:14.509661 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bdd1d-ce24-47ea-abff-e5b7a011f6e5 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:14.514262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455a3db-51f4-4e08-b960-004940d372cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:14.514860 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455a3db-51f4-4e08-b960-004940d372cb admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:14.515137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455a3db-51f4-4e08-b960-004940d372cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:14.515404 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a455a3db-51f4-4e08-b960-004940d372cb admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:14.516045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a455a3db-51f4-4e08-b960-004940d372cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:14.516342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a455a3db-51f4-4e08-b960-004940d372cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:14.517937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a455a3db-51f4-4e08-b960-004940d372cb admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:14.530332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1307/2624] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:14.534430 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dce673-7f84-4034-8116-d083b386dad2 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:14.544484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dce673-7f84-4034-8116-d083b386dad2 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:14.549490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e091519-a170-42f9-95b5-4ab62a0fbfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:14.550260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e091519-a170-42f9-95b5-4ab62a0fbfe8 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:14.550479 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e091519-a170-42f9-95b5-4ab62a0fbfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:14.550750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e091519-a170-42f9-95b5-4ab62a0fbfe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:14.551382 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e091519-a170-42f9-95b5-4ab62a0fbfe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:14.553306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e091519-a170-42f9-95b5-4ab62a0fbfe8 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:14.586507 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1318/2625] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:14 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:14.591521 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455a3db-51f4-4e08-b960-004940d372cb admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:14.604939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a455a3db-51f4-4e08-b960-004940d372cb admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:14.669927 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.686462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a833d2-4343-4e17-b8e7-4be10017e4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:14.687080 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a833d2-4343-4e17-b8e7-4be10017e4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:14.687499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a833d2-4343-4e17-b8e7-4be10017e4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:14.687817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a833d2-4343-4e17-b8e7-4be10017e4b3 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:14.688421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a833d2-4343-4e17-b8e7-4be10017e4b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:14.690559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a833d2-4343-4e17-b8e7-4be10017e4b3 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:14.715650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1308/2626] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:14 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:14.720094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e091519-a170-42f9-95b5-4ab62a0fbfe8 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:14.730707 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e091519-a170-42f9-95b5-4ab62a0fbfe8 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:14.735864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383f219-8783-47b6-8723-8678bb859435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:14.736534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383f219-8783-47b6-8723-8678bb859435 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:14.736811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383f219-8783-47b6-8723-8678bb859435 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:14.737080 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383f219-8783-47b6-8723-8678bb859435 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:14.737723 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383f219-8783-47b6-8723-8678bb859435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:14.739324 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383f219-8783-47b6-8723-8678bb859435 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:14.753965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1319/2627] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:14.755091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a833d2-4343-4e17-b8e7-4be10017e4b3 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:14.766115 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a833d2-4343-4e17-b8e7-4be10017e4b3 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:14.771738 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220477a-ec51-4257-a667-ab76ee2b8ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:14.772084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220477a-ec51-4257-a667-ab76ee2b8ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:14.772365 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220477a-ec51-4257-a667-ab76ee2b8ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:14.772641 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220477a-ec51-4257-a667-ab76ee2b8ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:14.773293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220477a-ec51-4257-a667-ab76ee2b8ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:14.775483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220477a-ec51-4257-a667-ab76ee2b8ee7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:14.790063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1309/2628] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:14.791450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383f219-8783-47b6-8723-8678bb859435 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:14.802814 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383f219-8783-47b6-8723-8678bb859435 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:14.807636 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89144f30-1c6b-4f99-b0ee-1443427b7b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:14.808295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89144f30-1c6b-4f99-b0ee-1443427b7b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:14.808605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89144f30-1c6b-4f99-b0ee-1443427b7b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:14.808882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89144f30-1c6b-4f99-b0ee-1443427b7b6e admin admin] PATH_INFO: `/v3/projects/1a378749bf5c463f8ee8d483b9f317c7/users/b28e2836c55645d2a185f6851b2a6a42/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:14.825956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89144f30-1c6b-4f99-b0ee-1443427b7b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a378749bf5c463f8ee8d483b9f317c7, user_id=b28e2836c55645d2a185f6851b2a6a42, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:14.828738 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89144f30-1c6b-4f99-b0ee-1443427b7b6e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:14.844988 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1320/2629] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:14 2022] PUT /identity/v3/projects/1a378749bf5c463f8ee8d483b9f317c7/users/b28e2836c55645d2a185f6851b2a6a42/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:14.856781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b83c9-7ec4-4f75-912c-b21056c2fb50 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:14.857167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b83c9-7ec4-4f75-912c-b21056c2fb50 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:14.857534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b83c9-7ec4-4f75-912c-b21056c2fb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:14.895617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d30b83c9-7ec4-4f75-912c-b21056c2fb50 None None] MFA Rules not processed for user `b28e2836c55645d2a185f6851b2a6a42`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:14.896751 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d30b83c9-7ec4-4f75-912c-b21056c2fb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:14.982693 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1310/2630] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:14.987783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89144f30-1c6b-4f99-b0ee-1443427b7b6e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:15.077567 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89144f30-1c6b-4f99-b0ee-1443427b7b6e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:15.082089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450b64d-f907-45ab-bfcb-e20cfc5a8bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:15.082691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450b64d-f907-45ab-bfcb-e20cfc5a8bae admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:15.082974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450b64d-f907-45ab-bfcb-e20cfc5a8bae admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:15.083403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450b64d-f907-45ab-bfcb-e20cfc5a8bae admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:15.084036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450b64d-f907-45ab-bfcb-e20cfc5a8bae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:15.085958 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450b64d-f907-45ab-bfcb-e20cfc5a8bae admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:15.114234 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1321/2631] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:14 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:15.119073 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b83c9-7ec4-4f75-912c-b21056c2fb50 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:15.130765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b83c9-7ec4-4f75-912c-b21056c2fb50 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:15.190186 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466c60b-9cd8-4883-878f-c5dafc27a2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:15.190803 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466c60b-9cd8-4883-878f-c5dafc27a2fa admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:15.191085 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466c60b-9cd8-4883-878f-c5dafc27a2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:15.191358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466c60b-9cd8-4883-878f-c5dafc27a2fa admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:15.192011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466c60b-9cd8-4883-878f-c5dafc27a2fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:15.194032 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466c60b-9cd8-4883-878f-c5dafc27a2fa admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:15.217178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1311/2632] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:15 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:15.221283 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450b64d-f907-45ab-bfcb-e20cfc5a8bae admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:15.233666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450b64d-f907-45ab-bfcb-e20cfc5a8bae admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:15.238689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468b58e-195c-47cd-812f-338996a1f60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:15.239328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468b58e-195c-47cd-812f-338996a1f60b admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:15.239601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468b58e-195c-47cd-812f-338996a1f60b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:15.239867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468b58e-195c-47cd-812f-338996a1f60b admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:15.240503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468b58e-195c-47cd-812f-338996a1f60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:15.242225 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468b58e-195c-47cd-812f-338996a1f60b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:15.253512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1322/2633] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:15.257690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466c60b-9cd8-4883-878f-c5dafc27a2fa admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:15.268892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466c60b-9cd8-4883-878f-c5dafc27a2fa admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:15.274070 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd3aed-8b76-44ad-9956-93c756bb084d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:15.275010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd3aed-8b76-44ad-9956-93c756bb084d admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:15.275291 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd3aed-8b76-44ad-9956-93c756bb084d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:15.275557 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd3aed-8b76-44ad-9956-93c756bb084d admin admin] PATH_INFO: `/v3/projects/578fd8e6cfee44fb89129235a9362db8/users/71e85402c31e48b294996fd6202d8783/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:15.290928 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd3aed-8b76-44ad-9956-93c756bb084d admin admin] RBAC: Authorizing `identity:create_grant(project_id=578fd8e6cfee44fb89129235a9362db8, user_id=71e85402c31e48b294996fd6202d8783, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:15.293664 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd3aed-8b76-44ad-9956-93c756bb084d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:15.307136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1312/2634] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:15 2022] PUT /identity/v3/projects/578fd8e6cfee44fb89129235a9362db8/users/71e85402c31e48b294996fd6202d8783/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:15.311331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468b58e-195c-47cd-812f-338996a1f60b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:15.322850 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468b58e-195c-47cd-812f-338996a1f60b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:15.380301 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6b14f-cc6e-4a0a-989c-84c002ae7b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:15.380906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6b14f-cc6e-4a0a-989c-84c002ae7b77 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:15.381182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6b14f-cc6e-4a0a-989c-84c002ae7b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:15.381457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6b14f-cc6e-4a0a-989c-84c002ae7b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:15.382124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6b14f-cc6e-4a0a-989c-84c002ae7b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:15.383701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6b14f-cc6e-4a0a-989c-84c002ae7b77 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:15.393244 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1323/2635] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:01:15.397552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd3aed-8b76-44ad-9956-93c756bb084d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:15.407763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd3aed-8b76-44ad-9956-93c756bb084d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:15.412415 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f7199-f3a4-44c9-a513-f3e9541917e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:15.413058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f7199-f3a4-44c9-a513-f3e9541917e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:15.413338 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f7199-f3a4-44c9-a513-f3e9541917e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:15.413611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f7199-f3a4-44c9-a513-f3e9541917e6 admin admin] PATH_INFO: `/v3/projects/578fd8e6cfee44fb89129235a9362db8/users/71e85402c31e48b294996fd6202d8783/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:15.416065 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f7199-f3a4-44c9-a513-f3e9541917e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=578fd8e6cfee44fb89129235a9362db8, user_id=71e85402c31e48b294996fd6202d8783, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:15.418746 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f7199-f3a4-44c9-a513-f3e9541917e6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:15.428691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1313/2636] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:15 2022] PUT /identity/v3/projects/578fd8e6cfee44fb89129235a9362db8/users/71e85402c31e48b294996fd6202d8783/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:15.439231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257f1f8-227c-4148-848a-8b56ecc86332 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:15.439596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257f1f8-227c-4148-848a-8b56ecc86332 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:15.439943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257f1f8-227c-4148-848a-8b56ecc86332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:15.459997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:15.471449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-2257f1f8-227c-4148-848a-8b56ecc86332 None None] MFA Rules not processed for user `71e85402c31e48b294996fd6202d8783`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:15.472795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2257f1f8-227c-4148-848a-8b56ecc86332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:15.544071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1324/2637] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:15.556547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f7199-f3a4-44c9-a513-f3e9541917e6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:15.558703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-002f7199-f3a4-44c9-a513-f3e9541917e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:15.569403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:15.626711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f7199-f3a4-44c9-a513-f3e9541917e6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:15.631700 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05a5bc-944f-43e6-a42d-cc46d2de88b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:15.632699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05a5bc-944f-43e6-a42d-cc46d2de88b2 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:15.633260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05a5bc-944f-43e6-a42d-cc46d2de88b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:15.633571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05a5bc-944f-43e6-a42d-cc46d2de88b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:15.691786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05a5bc-944f-43e6-a42d-cc46d2de88b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:15.693209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05a5bc-944f-43e6-a42d-cc46d2de88b2 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:15.701986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1314/2638] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:18.079046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8fa45-8035-4007-8773-f728c86d06c0 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:18.079441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8fa45-8035-4007-8773-f728c86d06c0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:18.079597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8fa45-8035-4007-8773-f728c86d06c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:18.101128 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-0ae8fa45-8035-4007-8773-f728c86d06c0 None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:18.102928 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0ae8fa45-8035-4007-8773-f728c86d06c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:18.166289 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1325/2639] 10.222.0.55 () {58 vars in 967 bytes} [Thu Sep 8 14:01:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:18.178836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05a5bc-944f-43e6-a42d-cc46d2de88b2 service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:18.180221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-dc05a5bc-944f-43e6-a42d-cc46d2de88b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:18.247835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05a5bc-944f-43e6-a42d-cc46d2de88b2 service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:18.252704 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d08e64-7d1b-47e6-bc03-dc97bba8d9f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:18.253311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d08e64-7d1b-47e6-bc03-dc97bba8d9f8 service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:18.253590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d08e64-7d1b-47e6-bc03-dc97bba8d9f8 service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:18.253900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d08e64-7d1b-47e6-bc03-dc97bba8d9f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:18.262079 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d08e64-7d1b-47e6-bc03-dc97bba8d9f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:18.263708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d08e64-7d1b-47e6-bc03-dc97bba8d9f8 service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:18.272243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1315/2640] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 14:01:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:01:22.492773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8fa45-8035-4007-8773-f728c86d06c0 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:22.566982 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8fa45-8035-4007-8773-f728c86d06c0 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:22.574736 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:22.575389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:22.575642 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:22.575919 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] PATH_INFO: `/v3/users/b28e2836c55645d2a185f6851b2a6a42` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:22.577062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b28e2836c55645d2a185f6851b2a6a42)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:22.579020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:22.623252 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:22.629137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b28e2836c55645d2a185f6851b2a6a42'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:22.637480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b28e2836c55645d2a185f6851b2a6a42'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:22.638128 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b28e2836c55645d2a185f6851b2a6a42'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:22.638797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b28e2836c55645d2a185f6851b2a6a42'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:22.646575 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b28e2836c55645d2a185f6851b2a6a42'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:22.665697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1326/2641] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:22 2022] DELETE /identity/v3/users/b28e2836c55645d2a185f6851b2a6a42 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:22.670351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ae858-f3de-42b7-97fe-4aef90f9611a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:22.670856 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ae858-f3de-42b7-97fe-4aef90f9611a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:22.670967 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ae858-f3de-42b7-97fe-4aef90f9611a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:22.694572 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-792ae858-f3de-42b7-97fe-4aef90f9611a None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:22.695827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-792ae858-f3de-42b7-97fe-4aef90f9611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:22.741763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:22.756636 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1316/2642] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:22.767670 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:22.770048 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:22.829986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e30484-50f9-483c-848e-84abcd544c38 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:22.834176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33eb13-afd2-48f0-a8b0-0560d363a4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:22.834826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33eb13-afd2-48f0-a8b0-0560d363a4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:22.835105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33eb13-afd2-48f0-a8b0-0560d363a4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:22.835372 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd33eb13-afd2-48f0-a8b0-0560d363a4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:22.845403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33eb13-afd2-48f0-a8b0-0560d363a4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:22.846906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd33eb13-afd2-48f0-a8b0-0560d363a4a1 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:22.857290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1327/2643] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:23.130086 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ae858-f3de-42b7-97fe-4aef90f9611a None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:23.131614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-792ae858-f3de-42b7-97fe-4aef90f9611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:23.144476 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ae858-f3de-42b7-97fe-4aef90f9611a None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:23.150311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:23.151108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:23.151592 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:23.151956 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] PATH_INFO: `/v3/projects/1a378749bf5c463f8ee8d483b9f317c7` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:23.153108 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a378749bf5c463f8ee8d483b9f317c7)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:23.155203 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:23.202774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a378749bf5c463f8ee8d483b9f317c7', 'request_id': 'req-25e16d59-689d-436c-95f5-12a503046044'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.208857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a378749bf5c463f8ee8d483b9f317c7', 'request_id': 'req-25e16d59-689d-436c-95f5-12a503046044'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.215151 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a378749bf5c463f8ee8d483b9f317c7', 'request_id': 'req-25e16d59-689d-436c-95f5-12a503046044'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.228343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] The token cache is being invalidate because project 1a378749bf5c463f8ee8d483b9f317c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:23.228628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.229258 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.230770 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1317/2644] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:23 2022] DELETE /identity/v3/projects/1a378749bf5c463f8ee8d483b9f317c7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:23.234771 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33eb13-afd2-48f0-a8b0-0560d363a4a1 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:23.244750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd33eb13-afd2-48f0-a8b0-0560d363a4a1 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:23.301892 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440139b1-6786-48c2-bce6-2f43fd6849a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:23.302522 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440139b1-6786-48c2-bce6-2f43fd6849a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:23.302801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440139b1-6786-48c2-bce6-2f43fd6849a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:23.303070 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440139b1-6786-48c2-bce6-2f43fd6849a2 admin admin] PATH_INFO: `/v3/users/71e85402c31e48b294996fd6202d8783` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:23.304300 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440139b1-6786-48c2-bce6-2f43fd6849a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71e85402c31e48b294996fd6202d8783)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:23.306776 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440139b1-6786-48c2-bce6-2f43fd6849a2 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:23.352835 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-440139b1-6786-48c2-bce6-2f43fd6849a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71e85402c31e48b294996fd6202d8783'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.359763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-440139b1-6786-48c2-bce6-2f43fd6849a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71e85402c31e48b294996fd6202d8783'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.360421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-440139b1-6786-48c2-bce6-2f43fd6849a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71e85402c31e48b294996fd6202d8783'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.361018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-440139b1-6786-48c2-bce6-2f43fd6849a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71e85402c31e48b294996fd6202d8783'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.367845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-440139b1-6786-48c2-bce6-2f43fd6849a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71e85402c31e48b294996fd6202d8783'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.368911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:23.387155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1328/2645] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:23 2022] DELETE /identity/v3/users/71e85402c31e48b294996fd6202d8783 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:23.662784 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:23.665161 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:23.678553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e16d59-689d-436c-95f5-12a503046044 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:23.743646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb5768-aadd-438a-a2ef-be42bce7df47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:23.744268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb5768-aadd-438a-a2ef-be42bce7df47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:23.744547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb5768-aadd-438a-a2ef-be42bce7df47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:23.744819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb5768-aadd-438a-a2ef-be42bce7df47 admin admin] PATH_INFO: `/v3/projects/578fd8e6cfee44fb89129235a9362db8` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:23.746015 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb5768-aadd-438a-a2ef-be42bce7df47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=578fd8e6cfee44fb89129235a9362db8)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:23.747946 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb5768-aadd-438a-a2ef-be42bce7df47 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:23.796060 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-93bb5768-aadd-438a-a2ef-be42bce7df47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '578fd8e6cfee44fb89129235a9362db8', 'request_id': 'req-93bb5768-aadd-438a-a2ef-be42bce7df47'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.803337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-93bb5768-aadd-438a-a2ef-be42bce7df47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '578fd8e6cfee44fb89129235a9362db8', 'request_id': 'req-93bb5768-aadd-438a-a2ef-be42bce7df47'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.810404 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-93bb5768-aadd-438a-a2ef-be42bce7df47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '578fd8e6cfee44fb89129235a9362db8', 'request_id': 'req-93bb5768-aadd-438a-a2ef-be42bce7df47'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.822939 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-93bb5768-aadd-438a-a2ef-be42bce7df47 admin admin] The token cache is being invalidate because project 578fd8e6cfee44fb89129235a9362db8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:23.823235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-93bb5768-aadd-438a-a2ef-be42bce7df47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.823857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-93bb5768-aadd-438a-a2ef-be42bce7df47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:23.825339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1318/2646] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:23 2022] DELETE /identity/v3/projects/578fd8e6cfee44fb89129235a9362db8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:23.830496 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbca581-8ff1-43a0-b10b-132c87a10101 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:23.830980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbca581-8ff1-43a0-b10b-132c87a10101 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:23.831203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbca581-8ff1-43a0-b10b-132c87a10101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:23.853156 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1bbca581-8ff1-43a0-b10b-132c87a10101 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:23.854373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1bbca581-8ff1-43a0-b10b-132c87a10101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:23.924214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1329/2647] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:23.932772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8abb52-fee7-4724-9d66-13a71c0adf99 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:23.933161 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8abb52-fee7-4724-9d66-13a71c0adf99 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:23.933456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8abb52-fee7-4724-9d66-13a71c0adf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:23.942148 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:23.955510 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-fa8abb52-fee7-4724-9d66-13a71c0adf99 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:23.957337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-fa8abb52-fee7-4724-9d66-13a71c0adf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:23.967483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1319/2648] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:23.971766 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbca581-8ff1-43a0-b10b-132c87a10101 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:23.982130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbca581-8ff1-43a0-b10b-132c87a10101 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:23.986725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770f8c9-b146-440b-9119-58ab121a9f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:23.987312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770f8c9-b146-440b-9119-58ab121a9f1b admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:23.987591 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770f8c9-b146-440b-9119-58ab121a9f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:23.987860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770f8c9-b146-440b-9119-58ab121a9f1b admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:23.988499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770f8c9-b146-440b-9119-58ab121a9f1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:23.988799 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770f8c9-b146-440b-9119-58ab121a9f1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:23.990746 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770f8c9-b146-440b-9119-58ab121a9f1b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:24.002155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1330/2649] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:24.006446 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8abb52-fee7-4724-9d66-13a71c0adf99 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:24.015566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8abb52-fee7-4724-9d66-13a71c0adf99 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:24.020160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908f165-1e01-4eae-a90e-055a5cbf89af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:24.020753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908f165-1e01-4eae-a90e-055a5cbf89af admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:24.021056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908f165-1e01-4eae-a90e-055a5cbf89af admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:24.021325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908f165-1e01-4eae-a90e-055a5cbf89af admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:24.021980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908f165-1e01-4eae-a90e-055a5cbf89af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:24.024346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908f165-1e01-4eae-a90e-055a5cbf89af admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:24.052585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1320/2650] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:24 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:24.057186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770f8c9-b146-440b-9119-58ab121a9f1b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:24.067442 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770f8c9-b146-440b-9119-58ab121a9f1b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:24.125337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a398e19-47e2-4cb0-82ec-4cee08b7db56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:24.125971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a398e19-47e2-4cb0-82ec-4cee08b7db56 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:24.126252 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a398e19-47e2-4cb0-82ec-4cee08b7db56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:24.126524 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a398e19-47e2-4cb0-82ec-4cee08b7db56 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:24.127144 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a398e19-47e2-4cb0-82ec-4cee08b7db56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:24.129031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a398e19-47e2-4cb0-82ec-4cee08b7db56 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:24.151072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1331/2651] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:24 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:24.155253 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908f165-1e01-4eae-a90e-055a5cbf89af admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:24.165901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908f165-1e01-4eae-a90e-055a5cbf89af admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:24.170436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a968f-13fa-49a1-944b-db3989a2df51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:24.171032 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a968f-13fa-49a1-944b-db3989a2df51 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:24.171311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a968f-13fa-49a1-944b-db3989a2df51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:24.171582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a968f-13fa-49a1-944b-db3989a2df51 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:24.172222 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a968f-13fa-49a1-944b-db3989a2df51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:24.173840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a968f-13fa-49a1-944b-db3989a2df51 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:24.183748 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1321/2652] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:24.187541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a398e19-47e2-4cb0-82ec-4cee08b7db56 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:24.197388 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a398e19-47e2-4cb0-82ec-4cee08b7db56 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:24.202240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745188f5-30b4-40c2-8a90-185af2f23167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:24.202885 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745188f5-30b4-40c2-8a90-185af2f23167 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:24.203212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745188f5-30b4-40c2-8a90-185af2f23167 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:24.203490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745188f5-30b4-40c2-8a90-185af2f23167 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:24.204133 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745188f5-30b4-40c2-8a90-185af2f23167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:24.205804 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745188f5-30b4-40c2-8a90-185af2f23167 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:24.217136 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1332/2653] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:24.221440 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a968f-13fa-49a1-944b-db3989a2df51 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:24.232699 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a968f-13fa-49a1-944b-db3989a2df51 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:24.237390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b044be-a4b2-4a11-b6fa-98924ca0d950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:24.238078 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b044be-a4b2-4a11-b6fa-98924ca0d950 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:24.238362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b044be-a4b2-4a11-b6fa-98924ca0d950 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:24.238640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b044be-a4b2-4a11-b6fa-98924ca0d950 admin admin] PATH_INFO: `/v3/projects/9bd62f19439e4bf9b7c3f392853092d5/users/5fdb1c6bf69a4e01b4303511d9d60e0a/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:24.253764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b044be-a4b2-4a11-b6fa-98924ca0d950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd62f19439e4bf9b7c3f392853092d5, user_id=5fdb1c6bf69a4e01b4303511d9d60e0a, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:24.256464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b044be-a4b2-4a11-b6fa-98924ca0d950 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:24.269608 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1322/2654] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:24 2022] PUT /identity/v3/projects/9bd62f19439e4bf9b7c3f392853092d5/users/5fdb1c6bf69a4e01b4303511d9d60e0a/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:24.281408 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a77d77-afdd-43c1-87ed-fb8949eac4ac None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:24.281719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a77d77-afdd-43c1-87ed-fb8949eac4ac None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:24.282075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a77d77-afdd-43c1-87ed-fb8949eac4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:24.310105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-19a77d77-afdd-43c1-87ed-fb8949eac4ac None None] MFA Rules not processed for user `5fdb1c6bf69a4e01b4303511d9d60e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:24.311359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-19a77d77-afdd-43c1-87ed-fb8949eac4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:24.346514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:24.381472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1333/2655] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:24.394553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b044be-a4b2-4a11-b6fa-98924ca0d950 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:24.396571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-48b044be-a4b2-4a11-b6fa-98924ca0d950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:24.454280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b044be-a4b2-4a11-b6fa-98924ca0d950 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:24.458429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebfa1e-b56a-44f1-acdb-e164c2f2ec95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:24.459126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebfa1e-b56a-44f1-acdb-e164c2f2ec95 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:24.459349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebfa1e-b56a-44f1-acdb-e164c2f2ec95 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:24.459686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebfa1e-b56a-44f1-acdb-e164c2f2ec95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:24.468874 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ebfa1e-b56a-44f1-acdb-e164c2f2ec95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:24.470329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ebfa1e-b56a-44f1-acdb-e164c2f2ec95 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:24.479302 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1323/2656] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:34.378320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=11 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:34.474996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:38.230071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a77d77-afdd-43c1-87ed-fb8949eac4ac None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.231636 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-19a77d77-afdd-43c1-87ed-fb8949eac4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:38.298653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a77d77-afdd-43c1-87ed-fb8949eac4ac None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.303117 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:38.303741 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.304020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.304294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] PATH_INFO: `/v3/users/66f89777d8d54397884aa1d553e921c4` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.305929 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66f89777d8d54397884aa1d553e921c4)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:38.307829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:38.355163 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66f89777d8d54397884aa1d553e921c4'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.363365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66f89777d8d54397884aa1d553e921c4'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.364021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66f89777d8d54397884aa1d553e921c4'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.364639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66f89777d8d54397884aa1d553e921c4'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.373064 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66f89777d8d54397884aa1d553e921c4'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.392650 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1334/2657] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:38 2022] DELETE /identity/v3/users/66f89777d8d54397884aa1d553e921c4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:38.397475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445c542-9d2c-4012-b272-a8d3cc9e7921 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.397997 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445c542-9d2c-4012-b272-a8d3cc9e7921 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.398099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2445c542-9d2c-4012-b272-a8d3cc9e7921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.416916 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2445c542-9d2c-4012-b272-a8d3cc9e7921 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:38.418138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2445c542-9d2c-4012-b272-a8d3cc9e7921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:38.479488 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1324/2658] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:38.489207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.491097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:38.548489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201fbad-9528-4995-a209-4d8e5ee73c59 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.553387 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6cb44-263d-4b18-b841-918bcac0e3b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:38.555403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6cb44-263d-4b18-b841-918bcac0e3b3 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.555403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6cb44-263d-4b18-b841-918bcac0e3b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.556135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6cb44-263d-4b18-b841-918bcac0e3b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.567509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6cb44-263d-4b18-b841-918bcac0e3b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:38.568945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6cb44-263d-4b18-b841-918bcac0e3b3 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:38.579137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1335/2659] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:38.842221 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445c542-9d2c-4012-b272-a8d3cc9e7921 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.843627 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2445c542-9d2c-4012-b272-a8d3cc9e7921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:38.856397 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2445c542-9d2c-4012-b272-a8d3cc9e7921 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.861270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:38.862121 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:38.862535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:38.862816 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] PATH_INFO: `/v3/projects/16bdb7cef22847fc8e05717b28bbedb4` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:38.864368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=16bdb7cef22847fc8e05717b28bbedb4)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:38.866357 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:38.913343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16bdb7cef22847fc8e05717b28bbedb4', 'request_id': 'req-997185d7-f1d1-446d-91a5-dfefb62f13fc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.920047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16bdb7cef22847fc8e05717b28bbedb4', 'request_id': 'req-997185d7-f1d1-446d-91a5-dfefb62f13fc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.925554 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16bdb7cef22847fc8e05717b28bbedb4', 'request_id': 'req-997185d7-f1d1-446d-91a5-dfefb62f13fc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.939042 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] The token cache is being invalidate because project 16bdb7cef22847fc8e05717b28bbedb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:38.939330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.939968 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:38.941445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1325/2660] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:38 2022] DELETE /identity/v3/projects/16bdb7cef22847fc8e05717b28bbedb4 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:38.945448 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6cb44-263d-4b18-b841-918bcac0e3b3 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:38.955053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6cb44-263d-4b18-b841-918bcac0e3b3 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:38.973796 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:39.016468 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1a0c6-0daf-4993-b301-5d8765b55e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.017083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1a0c6-0daf-4993-b301-5d8765b55e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.017411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1a0c6-0daf-4993-b301-5d8765b55e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.017635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1a0c6-0daf-4993-b301-5d8765b55e4f admin admin] PATH_INFO: `/v3/users/4c32d9ace6144d84bf511c82e7bc0ff6` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.018971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1a0c6-0daf-4993-b301-5d8765b55e4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c32d9ace6144d84bf511c82e7bc0ff6)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.020907 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1a0c6-0daf-4993-b301-5d8765b55e4f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.069024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d2e1a0c6-0daf-4993-b301-5d8765b55e4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c32d9ace6144d84bf511c82e7bc0ff6'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:39.076797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d2e1a0c6-0daf-4993-b301-5d8765b55e4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c32d9ace6144d84bf511c82e7bc0ff6'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:39.078035 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d2e1a0c6-0daf-4993-b301-5d8765b55e4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c32d9ace6144d84bf511c82e7bc0ff6'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:39.078736 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d2e1a0c6-0daf-4993-b301-5d8765b55e4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c32d9ace6144d84bf511c82e7bc0ff6'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:39.086581 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d2e1a0c6-0daf-4993-b301-5d8765b55e4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c32d9ace6144d84bf511c82e7bc0ff6'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:39.106919 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1336/2661] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:38 2022] DELETE /identity/v3/users/4c32d9ace6144d84bf511c82e7bc0ff6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:39.398509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:39.400486 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:39.419450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997185d7-f1d1-446d-91a5-dfefb62f13fc admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:39.479093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf2626-2d1c-4da2-b976-598a7b8abd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.479739 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf2626-2d1c-4da2-b976-598a7b8abd21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.480021 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf2626-2d1c-4da2-b976-598a7b8abd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.480297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf2626-2d1c-4da2-b976-598a7b8abd21 admin admin] PATH_INFO: `/v3/projects/b1a4083f55834b39a7d6f6487b7d187a` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.481419 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf2626-2d1c-4da2-b976-598a7b8abd21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1a4083f55834b39a7d6f6487b7d187a)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.483372 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf2626-2d1c-4da2-b976-598a7b8abd21 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.514774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:39.532959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2ebf2626-2d1c-4da2-b976-598a7b8abd21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1a4083f55834b39a7d6f6487b7d187a', 'request_id': 'req-2ebf2626-2d1c-4da2-b976-598a7b8abd21'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:39.539147 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2ebf2626-2d1c-4da2-b976-598a7b8abd21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1a4083f55834b39a7d6f6487b7d187a', 'request_id': 'req-2ebf2626-2d1c-4da2-b976-598a7b8abd21'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:39.544704 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2ebf2626-2d1c-4da2-b976-598a7b8abd21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1a4083f55834b39a7d6f6487b7d187a', 'request_id': 'req-2ebf2626-2d1c-4da2-b976-598a7b8abd21'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:39.557235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2ebf2626-2d1c-4da2-b976-598a7b8abd21 admin admin] The token cache is being invalidate because project b1a4083f55834b39a7d6f6487b7d187a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:39.557569 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2ebf2626-2d1c-4da2-b976-598a7b8abd21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:39.558383 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2ebf2626-2d1c-4da2-b976-598a7b8abd21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:39.560413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1326/2662] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:39 2022] DELETE /identity/v3/projects/b1a4083f55834b39a7d6f6487b7d187a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:39.565572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3a429-d355-4afd-9807-f609e2917aba None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.566103 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3a429-d355-4afd-9807-f609e2917aba None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.566248 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3a429-d355-4afd-9807-f609e2917aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.585969 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-c7f3a429-d355-4afd-9807-f609e2917aba None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:39.587694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-c7f3a429-d355-4afd-9807-f609e2917aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:39.644232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1337/2663] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:39.654262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35375168-e92d-4b17-9b79-c217a24c24ee None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.654551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35375168-e92d-4b17-9b79-c217a24c24ee None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.654828 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35375168-e92d-4b17-9b79-c217a24c24ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.674230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-35375168-e92d-4b17-9b79-c217a24c24ee None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:39.675423 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-35375168-e92d-4b17-9b79-c217a24c24ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:39.683287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1327/2664] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:39.688081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3a429-d355-4afd-9807-f609e2917aba None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:39.699118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3a429-d355-4afd-9807-f609e2917aba None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:39.703711 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc515b26-1816-4b46-b672-0d8000d3d219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.704308 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc515b26-1816-4b46-b672-0d8000d3d219 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.704590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc515b26-1816-4b46-b672-0d8000d3d219 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.704863 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc515b26-1816-4b46-b672-0d8000d3d219 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.705505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc515b26-1816-4b46-b672-0d8000d3d219 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:39.705845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc515b26-1816-4b46-b672-0d8000d3d219 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.707425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc515b26-1816-4b46-b672-0d8000d3d219 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.719710 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1338/2665] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:39.723992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35375168-e92d-4b17-9b79-c217a24c24ee None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:39.732999 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35375168-e92d-4b17-9b79-c217a24c24ee None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:39.737545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ccaae-78ff-4038-a797-b1ab196ba110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.738170 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ccaae-78ff-4038-a797-b1ab196ba110 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.738445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ccaae-78ff-4038-a797-b1ab196ba110 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.738714 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ccaae-78ff-4038-a797-b1ab196ba110 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.739331 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ccaae-78ff-4038-a797-b1ab196ba110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.741189 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ccaae-78ff-4038-a797-b1ab196ba110 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.770006 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1328/2666] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:39 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:39.774162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc515b26-1816-4b46-b672-0d8000d3d219 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:39.784250 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc515b26-1816-4b46-b672-0d8000d3d219 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:39.810207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:39.842943 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c466c-7043-48ac-91cf-0dd24751ba00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.843543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c466c-7043-48ac-91cf-0dd24751ba00 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.843823 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c466c-7043-48ac-91cf-0dd24751ba00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.844094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c466c-7043-48ac-91cf-0dd24751ba00 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.844714 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c466c-7043-48ac-91cf-0dd24751ba00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.846764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c466c-7043-48ac-91cf-0dd24751ba00 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.867095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1339/2667] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:39 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:39.871749 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ccaae-78ff-4038-a797-b1ab196ba110 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:39.881751 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ccaae-78ff-4038-a797-b1ab196ba110 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:39.886817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ea672-d142-4cf9-932e-a6e4e885fb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.887407 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ea672-d142-4cf9-932e-a6e4e885fb5f admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.887680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ea672-d142-4cf9-932e-a6e4e885fb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.887953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ea672-d142-4cf9-932e-a6e4e885fb5f admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.888582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ea672-d142-4cf9-932e-a6e4e885fb5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.890190 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ea672-d142-4cf9-932e-a6e4e885fb5f admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.900068 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1329/2668] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:39.903949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c466c-7043-48ac-91cf-0dd24751ba00 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:39.914374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c466c-7043-48ac-91cf-0dd24751ba00 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:39.918984 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf8345-aa1c-429b-9119-aa482e2457fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.919570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf8345-aa1c-429b-9119-aa482e2457fe admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.919845 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf8345-aa1c-429b-9119-aa482e2457fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.920112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf8345-aa1c-429b-9119-aa482e2457fe admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.920747 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf8345-aa1c-429b-9119-aa482e2457fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.922346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf8345-aa1c-429b-9119-aa482e2457fe admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.933268 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1340/2669] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:39.937382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ea672-d142-4cf9-932e-a6e4e885fb5f admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:39.947963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ea672-d142-4cf9-932e-a6e4e885fb5f admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:39.952490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00db6f-9f96-4008-9a96-7ba2ce87bba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:39.953166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00db6f-9f96-4008-9a96-7ba2ce87bba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.953441 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00db6f-9f96-4008-9a96-7ba2ce87bba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.953709 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00db6f-9f96-4008-9a96-7ba2ce87bba2 admin admin] PATH_INFO: `/v3/projects/633dfbdc2c4744f4aa582babff5de160/users/a94b26aabfb3444d97dd5909515e0528/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:39.969127 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00db6f-9f96-4008-9a96-7ba2ce87bba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=633dfbdc2c4744f4aa582babff5de160, user_id=a94b26aabfb3444d97dd5909515e0528, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:39.971821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00db6f-9f96-4008-9a96-7ba2ce87bba2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:39.984768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1330/2670] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:39 2022] PUT /identity/v3/projects/633dfbdc2c4744f4aa582babff5de160/users/a94b26aabfb3444d97dd5909515e0528/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:39.995092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb195a1-2fbe-4e24-b870-f5d2cfcfceab None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:39.995400 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb195a1-2fbe-4e24-b870-f5d2cfcfceab None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:39.995674 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb195a1-2fbe-4e24-b870-f5d2cfcfceab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:40.022679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-cbb195a1-2fbe-4e24-b870-f5d2cfcfceab None None] MFA Rules not processed for user `a94b26aabfb3444d97dd5909515e0528`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:40.024415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-cbb195a1-2fbe-4e24-b870-f5d2cfcfceab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:40.087059 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1341/2671] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:40.091760 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00db6f-9f96-4008-9a96-7ba2ce87bba2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:40.152860 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00db6f-9f96-4008-9a96-7ba2ce87bba2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:40.157485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9138bab-9182-4545-b024-efd8c3af1021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:40.158112 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9138bab-9182-4545-b024-efd8c3af1021 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:40.158399 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9138bab-9182-4545-b024-efd8c3af1021 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:40.158672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9138bab-9182-4545-b024-efd8c3af1021 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:40.159298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9138bab-9182-4545-b024-efd8c3af1021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:40.161183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9138bab-9182-4545-b024-efd8c3af1021 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:40.186355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1331/2672] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:40 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:40.190493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb195a1-2fbe-4e24-b870-f5d2cfcfceab None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:40.201569 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb195a1-2fbe-4e24-b870-f5d2cfcfceab None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:40.259016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da9dfa-4faf-4146-a4d5-12a6c2187da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:40.259614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da9dfa-4faf-4146-a4d5-12a6c2187da6 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:40.259889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da9dfa-4faf-4146-a4d5-12a6c2187da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:40.260154 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da9dfa-4faf-4146-a4d5-12a6c2187da6 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:40.260775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da9dfa-4faf-4146-a4d5-12a6c2187da6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:40.262677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da9dfa-4faf-4146-a4d5-12a6c2187da6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:40.283116 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1342/2673] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:40 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:40.287152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9138bab-9182-4545-b024-efd8c3af1021 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:40.297305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9138bab-9182-4545-b024-efd8c3af1021 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:40.301979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a1f59-d4a1-45a7-a81f-3c5d06e3a92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:40.302579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a1f59-d4a1-45a7-a81f-3c5d06e3a92b admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:40.302863 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a1f59-d4a1-45a7-a81f-3c5d06e3a92b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:40.303138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597a1f59-d4a1-45a7-a81f-3c5d06e3a92b admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:40.303810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a1f59-d4a1-45a7-a81f-3c5d06e3a92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:40.305405 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597a1f59-d4a1-45a7-a81f-3c5d06e3a92b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:40.315819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1332/2674] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:40.319711 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da9dfa-4faf-4146-a4d5-12a6c2187da6 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:40.329975 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da9dfa-4faf-4146-a4d5-12a6c2187da6 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:40.334498 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215f364-79ec-4064-a3f8-4b4b07d4ff92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:40.335137 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215f364-79ec-4064-a3f8-4b4b07d4ff92 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:40.335413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215f364-79ec-4064-a3f8-4b4b07d4ff92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:40.335684 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215f364-79ec-4064-a3f8-4b4b07d4ff92 admin admin] PATH_INFO: `/v3/projects/9139c02f02c646e4b042b90b545bf003/users/24fffdc597d9448fab597aba7e9104e9/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:40.351351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215f364-79ec-4064-a3f8-4b4b07d4ff92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9139c02f02c646e4b042b90b545bf003, user_id=24fffdc597d9448fab597aba7e9104e9, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:40.353946 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3215f364-79ec-4064-a3f8-4b4b07d4ff92 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:40.367164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1343/2675] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:40 2022] PUT /identity/v3/projects/9139c02f02c646e4b042b90b545bf003/users/24fffdc597d9448fab597aba7e9104e9/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:40.371035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a1f59-d4a1-45a7-a81f-3c5d06e3a92b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:40.380889 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597a1f59-d4a1-45a7-a81f-3c5d06e3a92b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:40.401617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:40.439123 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d8ace-b6ae-4248-a862-18801694e583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:40.439727 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d8ace-b6ae-4248-a862-18801694e583 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:40.440008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d8ace-b6ae-4248-a862-18801694e583 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:40.440281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d8ace-b6ae-4248-a862-18801694e583 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:40.440914 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d8ace-b6ae-4248-a862-18801694e583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:40.442545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d8ace-b6ae-4248-a862-18801694e583 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:40.452705 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1333/2676] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:40.456989 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215f364-79ec-4064-a3f8-4b4b07d4ff92 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:40.463110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:40.469912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3215f364-79ec-4064-a3f8-4b4b07d4ff92 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:40.474981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ba0c0-2259-48ed-b033-4a7332bc3170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:40.475634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ba0c0-2259-48ed-b033-4a7332bc3170 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:40.475913 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ba0c0-2259-48ed-b033-4a7332bc3170 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:40.476186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ba0c0-2259-48ed-b033-4a7332bc3170 admin admin] PATH_INFO: `/v3/projects/9139c02f02c646e4b042b90b545bf003/users/24fffdc597d9448fab597aba7e9104e9/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:40.478184 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ba0c0-2259-48ed-b033-4a7332bc3170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9139c02f02c646e4b042b90b545bf003, user_id=24fffdc597d9448fab597aba7e9104e9, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:40.480827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1ba0c0-2259-48ed-b033-4a7332bc3170 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:40.490903 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1344/2677] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:40 2022] PUT /identity/v3/projects/9139c02f02c646e4b042b90b545bf003/users/24fffdc597d9448fab597aba7e9104e9/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:40.502258 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741cd86-efab-42ac-8764-15e85f55a1d6 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:40.502549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741cd86-efab-42ac-8764-15e85f55a1d6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:40.502827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741cd86-efab-42ac-8764-15e85f55a1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:40.531477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-8741cd86-efab-42ac-8764-15e85f55a1d6 None None] MFA Rules not processed for user `24fffdc597d9448fab597aba7e9104e9`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:40.532672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8741cd86-efab-42ac-8764-15e85f55a1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:40.600941 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1334/2678] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:40.612349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ba0c0-2259-48ed-b033-4a7332bc3170 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:40.614120 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-fe1ba0c0-2259-48ed-b033-4a7332bc3170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:40.671596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ba0c0-2259-48ed-b033-4a7332bc3170 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:40.675227 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feab8cd-70c5-4e30-8f54-6deb26f99192 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:40.675822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feab8cd-70c5-4e30-8f54-6deb26f99192 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:40.676100 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feab8cd-70c5-4e30-8f54-6deb26f99192 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:40.676371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feab8cd-70c5-4e30-8f54-6deb26f99192 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:40.731295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feab8cd-70c5-4e30-8f54-6deb26f99192 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:40.732713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feab8cd-70c5-4e30-8f54-6deb26f99192 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:40.741392 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1345/2679] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:47.475606 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741cd86-efab-42ac-8764-15e85f55a1d6 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:47.486154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741cd86-efab-42ac-8764-15e85f55a1d6 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:47.490952 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f79195-a498-4b94-972c-ab991c5787a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:47.491564 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f79195-a498-4b94-972c-ab991c5787a4 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:47.491846 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f79195-a498-4b94-972c-ab991c5787a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:47.492171 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f79195-a498-4b94-972c-ab991c5787a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:47.501373 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f79195-a498-4b94-972c-ab991c5787a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:47.502835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f79195-a498-4b94-972c-ab991c5787a4 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:47.512311 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1335/2680] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:50.737405 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:53.558441 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feab8cd-70c5-4e30-8f54-6deb26f99192 service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:53.560002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2feab8cd-70c5-4e30-8f54-6deb26f99192 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:53.623868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feab8cd-70c5-4e30-8f54-6deb26f99192 service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:53.628319 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:53.628936 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.629212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.629482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] PATH_INFO: `/v3/users/5fdb1c6bf69a4e01b4303511d9d60e0a` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.630624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fdb1c6bf69a4e01b4303511d9d60e0a)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:53.632612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:53.677192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fdb1c6bf69a4e01b4303511d9d60e0a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:53.685303 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fdb1c6bf69a4e01b4303511d9d60e0a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:53.685948 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fdb1c6bf69a4e01b4303511d9d60e0a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:53.686569 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fdb1c6bf69a4e01b4303511d9d60e0a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:53.693612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fdb1c6bf69a4e01b4303511d9d60e0a'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:53.710493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1346/2681] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:53 2022] DELETE /identity/v3/users/5fdb1c6bf69a4e01b4303511d9d60e0a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:53.715351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f2275-6eca-4c59-9291-6d2f6fad5670 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.715822 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f2275-6eca-4c59-9291-6d2f6fad5670 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.715922 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f2275-6eca-4c59-9291-6d2f6fad5670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.735010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2f6f2275-6eca-4c59-9291-6d2f6fad5670 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:53.736215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2f6f2275-6eca-4c59-9291-6d2f6fad5670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:53.794187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1336/2682] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:53.803939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:53.805915 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:53.862580 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e212d-acc4-4008-bdfb-e32697b1dc65 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:53.866218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182e1f8-9be8-491c-8c8f-b3214389a604 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:53.866881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182e1f8-9be8-491c-8c8f-b3214389a604 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.867651 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182e1f8-9be8-491c-8c8f-b3214389a604 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.867925 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182e1f8-9be8-491c-8c8f-b3214389a604 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.879090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182e1f8-9be8-491c-8c8f-b3214389a604 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:53.880551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182e1f8-9be8-491c-8c8f-b3214389a604 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:53.891391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1347/2683] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:54.159362 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f2275-6eca-4c59-9291-6d2f6fad5670 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:54.160811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2f6f2275-6eca-4c59-9291-6d2f6fad5670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:54.173347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f2275-6eca-4c59-9291-6d2f6fad5670 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:54.178328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f60dae-b516-4f43-a8a4-4f66cf3be098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:54.178987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f60dae-b516-4f43-a8a4-4f66cf3be098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.179254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f60dae-b516-4f43-a8a4-4f66cf3be098 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.180081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f60dae-b516-4f43-a8a4-4f66cf3be098 admin admin] PATH_INFO: `/v3/projects/9bd62f19439e4bf9b7c3f392853092d5` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.189326 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f60dae-b516-4f43-a8a4-4f66cf3be098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd62f19439e4bf9b7c3f392853092d5)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:54.191540 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f60dae-b516-4f43-a8a4-4f66cf3be098 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:54.225943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:54.241950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-55f60dae-b516-4f43-a8a4-4f66cf3be098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd62f19439e4bf9b7c3f392853092d5', 'request_id': 'req-55f60dae-b516-4f43-a8a4-4f66cf3be098'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.248144 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-55f60dae-b516-4f43-a8a4-4f66cf3be098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd62f19439e4bf9b7c3f392853092d5', 'request_id': 'req-55f60dae-b516-4f43-a8a4-4f66cf3be098'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.253749 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-55f60dae-b516-4f43-a8a4-4f66cf3be098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd62f19439e4bf9b7c3f392853092d5', 'request_id': 'req-55f60dae-b516-4f43-a8a4-4f66cf3be098'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.266089 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-55f60dae-b516-4f43-a8a4-4f66cf3be098 admin admin] The token cache is being invalidate because project 9bd62f19439e4bf9b7c3f392853092d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:54.266403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-55f60dae-b516-4f43-a8a4-4f66cf3be098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.267055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-55f60dae-b516-4f43-a8a4-4f66cf3be098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:54.268536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1337/2684] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:54 2022] DELETE /identity/v3/projects/9bd62f19439e4bf9b7c3f392853092d5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:54.274114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504b8f3-2f85-410a-989e-1ae687c0da14 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.274402 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504b8f3-2f85-410a-989e-1ae687c0da14 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.274680 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504b8f3-2f85-410a-989e-1ae687c0da14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.294596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4504b8f3-2f85-410a-989e-1ae687c0da14 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:54.295905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4504b8f3-2f85-410a-989e-1ae687c0da14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:54.312876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:54.358209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1348/2685] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:54.365712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d9730-9639-4347-bffe-80af5faeb2c7 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.366052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d9730-9639-4347-bffe-80af5faeb2c7 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.366334 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d9730-9639-4347-bffe-80af5faeb2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.386571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6b8d9730-9639-4347-bffe-80af5faeb2c7 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:54.387772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6b8d9730-9639-4347-bffe-80af5faeb2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:54.395971 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1338/2686] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:54.400626 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504b8f3-2f85-410a-989e-1ae687c0da14 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:54.413777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504b8f3-2f85-410a-989e-1ae687c0da14 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:54.414697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bba08-5092-4a65-9bef-7aab668ddd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:54.415298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bba08-5092-4a65-9bef-7aab668ddd83 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.415578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bba08-5092-4a65-9bef-7aab668ddd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.415848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bba08-5092-4a65-9bef-7aab668ddd83 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.416493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bba08-5092-4a65-9bef-7aab668ddd83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:54.416797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bba08-5092-4a65-9bef-7aab668ddd83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:54.418411 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bba08-5092-4a65-9bef-7aab668ddd83 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:54.430051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1349/2687] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:01:54.434307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d9730-9639-4347-bffe-80af5faeb2c7 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:54.443425 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d9730-9639-4347-bffe-80af5faeb2c7 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:54.448508 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b882a-69e5-4ad9-ae1b-242a3903bb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:54.449101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b882a-69e5-4ad9-ae1b-242a3903bb32 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.449379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b882a-69e5-4ad9-ae1b-242a3903bb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.449651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b882a-69e5-4ad9-ae1b-242a3903bb32 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.450321 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b882a-69e5-4ad9-ae1b-242a3903bb32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:54.452310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b882a-69e5-4ad9-ae1b-242a3903bb32 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:54.480237 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1339/2688] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:54 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:54.484458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bba08-5092-4a65-9bef-7aab668ddd83 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:54.495198 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bba08-5092-4a65-9bef-7aab668ddd83 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:54.549704 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff77def-d2ae-41ac-9789-00e1f2dca8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:54.550328 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff77def-d2ae-41ac-9789-00e1f2dca8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.550608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff77def-d2ae-41ac-9789-00e1f2dca8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.550879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff77def-d2ae-41ac-9789-00e1f2dca8f9 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.551497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff77def-d2ae-41ac-9789-00e1f2dca8f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:54.553398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff77def-d2ae-41ac-9789-00e1f2dca8f9 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:54.575247 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1350/2689] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:54 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:54.579499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b882a-69e5-4ad9-ae1b-242a3903bb32 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:54.589577 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b882a-69e5-4ad9-ae1b-242a3903bb32 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:54.594495 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2ddaf-ea1b-4ca0-bc8c-736e04737409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:54.595088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2ddaf-ea1b-4ca0-bc8c-736e04737409 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.595368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2ddaf-ea1b-4ca0-bc8c-736e04737409 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.595644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2ddaf-ea1b-4ca0-bc8c-736e04737409 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.596285 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2ddaf-ea1b-4ca0-bc8c-736e04737409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:54.597919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2ddaf-ea1b-4ca0-bc8c-736e04737409 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:54.608140 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1340/2690] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:54.611792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff77def-d2ae-41ac-9789-00e1f2dca8f9 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:54.622228 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff77def-d2ae-41ac-9789-00e1f2dca8f9 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:54.626796 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8364776-2da9-4276-a704-12fd615ea52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:54.627378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8364776-2da9-4276-a704-12fd615ea52e admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.627657 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8364776-2da9-4276-a704-12fd615ea52e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.627929 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8364776-2da9-4276-a704-12fd615ea52e admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.628563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8364776-2da9-4276-a704-12fd615ea52e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:54.630176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8364776-2da9-4276-a704-12fd615ea52e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:54.640474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1351/2691] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:54.644333 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2ddaf-ea1b-4ca0-bc8c-736e04737409 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:54.654074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2ddaf-ea1b-4ca0-bc8c-736e04737409 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:54.659585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3229b-43ca-4efa-8cef-b1ded54e09f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:54.660217 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3229b-43ca-4efa-8cef-b1ded54e09f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.660498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3229b-43ca-4efa-8cef-b1ded54e09f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.660771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3229b-43ca-4efa-8cef-b1ded54e09f6 admin admin] PATH_INFO: `/v3/projects/c8b3129643044cc2b299898627b5a8f4/users/ee0a49e7d8e24898b82c0510e06a3574/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.675450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3229b-43ca-4efa-8cef-b1ded54e09f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8b3129643044cc2b299898627b5a8f4, user_id=ee0a49e7d8e24898b82c0510e06a3574, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:54.678136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3229b-43ca-4efa-8cef-b1ded54e09f6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:54.691655 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1341/2692] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:54 2022] PUT /identity/v3/projects/c8b3129643044cc2b299898627b5a8f4/users/ee0a49e7d8e24898b82c0510e06a3574/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:54.757912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec1ab0-bfc4-4b25-8074-353e10f45e0b None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.758132 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec1ab0-bfc4-4b25-8074-353e10f45e0b None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.758360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec1ab0-bfc4-4b25-8074-353e10f45e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.787503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-a9ec1ab0-bfc4-4b25-8074-353e10f45e0b None None] MFA Rules not processed for user `ee0a49e7d8e24898b82c0510e06a3574`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:54.789239 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a9ec1ab0-bfc4-4b25-8074-353e10f45e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:54.853217 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1352/2693] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:01:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:54.864057 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3229b-43ca-4efa-8cef-b1ded54e09f6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:54.865870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5ca3229b-43ca-4efa-8cef-b1ded54e09f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:54.924623 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3229b-43ca-4efa-8cef-b1ded54e09f6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:54.928273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622e9f6-e57c-4bbf-a399-0b720d70edd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:54.928882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622e9f6-e57c-4bbf-a399-0b720d70edd5 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.929167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622e9f6-e57c-4bbf-a399-0b720d70edd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.929442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622e9f6-e57c-4bbf-a399-0b720d70edd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.937574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622e9f6-e57c-4bbf-a399-0b720d70edd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:54.939035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622e9f6-e57c-4bbf-a399-0b720d70edd5 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:54.947680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1342/2694] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:04.849663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:04.943681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:18.477764 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec1ab0-bfc4-4b25-8074-353e10f45e0b None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:18.542205 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec1ab0-bfc4-4b25-8074-353e10f45e0b None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:18.546815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:18.547437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:18.547713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:18.547979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] PATH_INFO: `/v3/users/ee0a49e7d8e24898b82c0510e06a3574` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:18.549108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee0a49e7d8e24898b82c0510e06a3574)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:18.551089 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:18.596606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee0a49e7d8e24898b82c0510e06a3574'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:18.604157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee0a49e7d8e24898b82c0510e06a3574'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:18.604765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee0a49e7d8e24898b82c0510e06a3574'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:18.605367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee0a49e7d8e24898b82c0510e06a3574'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:18.612150 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee0a49e7d8e24898b82c0510e06a3574'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:18.629152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1353/2695] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:18 2022] DELETE /identity/v3/users/ee0a49e7d8e24898b82c0510e06a3574 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:18.633633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a8359-a404-4ef0-a796-4ac779934be5 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:18.634163 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a8359-a404-4ef0-a796-4ac779934be5 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:18.634265 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3a8359-a404-4ef0-a796-4ac779934be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:18.654487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2b3a8359-a404-4ef0-a796-4ac779934be5 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:18.655703 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2b3a8359-a404-4ef0-a796-4ac779934be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:18.717898 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1343/2696] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:18.727586 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:18.729021 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:18.788613 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71deae33-fc8a-49d2-8499-16d595517904 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:18.792311 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05781919-ffe7-4cd0-8331-653385a94f74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:18.792909 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781919-ffe7-4cd0-8331-653385a94f74 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:18.793189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781919-ffe7-4cd0-8331-653385a94f74 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:18.793461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05781919-ffe7-4cd0-8331-653385a94f74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:18.802833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05781919-ffe7-4cd0-8331-653385a94f74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:18.804290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05781919-ffe7-4cd0-8331-653385a94f74 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:18.814753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1354/2697] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:19.098696 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a8359-a404-4ef0-a796-4ac779934be5 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:19.100266 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2b3a8359-a404-4ef0-a796-4ac779934be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:19.114441 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3a8359-a404-4ef0-a796-4ac779934be5 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:19.119863 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0527af3-1056-44e6-8415-c101648d7fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:19.120512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0527af3-1056-44e6-8415-c101648d7fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.120791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0527af3-1056-44e6-8415-c101648d7fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.121064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0527af3-1056-44e6-8415-c101648d7fc6 admin admin] PATH_INFO: `/v3/projects/c8b3129643044cc2b299898627b5a8f4` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.122563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0527af3-1056-44e6-8415-c101648d7fc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8b3129643044cc2b299898627b5a8f4)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:19.125581 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0527af3-1056-44e6-8415-c101648d7fc6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:19.175285 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b0527af3-1056-44e6-8415-c101648d7fc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8b3129643044cc2b299898627b5a8f4', 'request_id': 'req-b0527af3-1056-44e6-8415-c101648d7fc6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.181993 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b0527af3-1056-44e6-8415-c101648d7fc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8b3129643044cc2b299898627b5a8f4', 'request_id': 'req-b0527af3-1056-44e6-8415-c101648d7fc6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.189084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b0527af3-1056-44e6-8415-c101648d7fc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8b3129643044cc2b299898627b5a8f4', 'request_id': 'req-b0527af3-1056-44e6-8415-c101648d7fc6'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.203422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b0527af3-1056-44e6-8415-c101648d7fc6 admin admin] The token cache is being invalidate because project c8b3129643044cc2b299898627b5a8f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:19.203722 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b0527af3-1056-44e6-8415-c101648d7fc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.204332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b0527af3-1056-44e6-8415-c101648d7fc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.206001 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1344/2698] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:19 2022] DELETE /identity/v3/projects/c8b3129643044cc2b299898627b5a8f4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:19.210979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72520991-b0c6-43d8-a1b6-e2045b23b5ba None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.211266 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72520991-b0c6-43d8-a1b6-e2045b23b5ba None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.211542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72520991-b0c6-43d8-a1b6-e2045b23b5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.232442 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-72520991-b0c6-43d8-a1b6-e2045b23b5ba None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:19.233625 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-72520991-b0c6-43d8-a1b6-e2045b23b5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:19.249940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:19.292829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1355/2699] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:19.300793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70656a-3130-4eac-b843-6658e1ef9239 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.301084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70656a-3130-4eac-b843-6658e1ef9239 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.301363 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70656a-3130-4eac-b843-6658e1ef9239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.320612 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-9f70656a-3130-4eac-b843-6658e1ef9239 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:19.321830 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9f70656a-3130-4eac-b843-6658e1ef9239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:19.330059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1345/2700] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:19.334268 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72520991-b0c6-43d8-a1b6-e2045b23b5ba None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:19.344512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72520991-b0c6-43d8-a1b6-e2045b23b5ba None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:19.349017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f947c-1869-481b-94f6-9d72b605edeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:19.349601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f947c-1869-481b-94f6-9d72b605edeb admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.349980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f947c-1869-481b-94f6-9d72b605edeb admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.350261 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f947c-1869-481b-94f6-9d72b605edeb admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.350884 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f947c-1869-481b-94f6-9d72b605edeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:19.351178 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f947c-1869-481b-94f6-9d72b605edeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:19.352739 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f947c-1869-481b-94f6-9d72b605edeb admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:19.364193 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1356/2701] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:19.368526 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70656a-3130-4eac-b843-6658e1ef9239 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:19.377662 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70656a-3130-4eac-b843-6658e1ef9239 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:19.382462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ee9d2-7085-4bcf-b4c1-eaea9d00156b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:19.383198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ee9d2-7085-4bcf-b4c1-eaea9d00156b admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.383970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ee9d2-7085-4bcf-b4c1-eaea9d00156b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.383970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ee9d2-7085-4bcf-b4c1-eaea9d00156b admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.385141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ee9d2-7085-4bcf-b4c1-eaea9d00156b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:19.387255 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8ee9d2-7085-4bcf-b4c1-eaea9d00156b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:19.414514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1346/2702] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:19 2022] POST /identity/v3/projects => generated 397 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:19.419230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f947c-1869-481b-94f6-9d72b605edeb admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:19.429240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f947c-1869-481b-94f6-9d72b605edeb admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:19.488141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573a362-a3c9-4f89-9172-b8cc7e839129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:19.488874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573a362-a3c9-4f89-9172-b8cc7e839129 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.489080 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573a362-a3c9-4f89-9172-b8cc7e839129 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.489368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573a362-a3c9-4f89-9172-b8cc7e839129 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.490075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573a362-a3c9-4f89-9172-b8cc7e839129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:19.491974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573a362-a3c9-4f89-9172-b8cc7e839129 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:19.513490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1357/2703] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:19 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:19.517630 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ee9d2-7085-4bcf-b4c1-eaea9d00156b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:19.527721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8ee9d2-7085-4bcf-b4c1-eaea9d00156b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:19.532264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862ac00-721f-4f8a-9f73-b3b36604fc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:19.532857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862ac00-721f-4f8a-9f73-b3b36604fc43 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.533135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862ac00-721f-4f8a-9f73-b3b36604fc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.533406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862ac00-721f-4f8a-9f73-b3b36604fc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.534084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862ac00-721f-4f8a-9f73-b3b36604fc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:19.535673 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862ac00-721f-4f8a-9f73-b3b36604fc43 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:19.546563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1347/2704] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:19.550550 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573a362-a3c9-4f89-9172-b8cc7e839129 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:19.561182 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573a362-a3c9-4f89-9172-b8cc7e839129 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:19.565881 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1332c-87f7-4781-87f5-0a2929120c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:19.566475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1332c-87f7-4781-87f5-0a2929120c36 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.566752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1332c-87f7-4781-87f5-0a2929120c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.567017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1332c-87f7-4781-87f5-0a2929120c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.567679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1332c-87f7-4781-87f5-0a2929120c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:19.569253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1332c-87f7-4781-87f5-0a2929120c36 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:19.580339 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1358/2705] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:19.584597 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862ac00-721f-4f8a-9f73-b3b36604fc43 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:19.594921 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862ac00-721f-4f8a-9f73-b3b36604fc43 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:19.599553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b24f3-6e8c-43b9-a904-ca40a9127b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:19.600196 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b24f3-6e8c-43b9-a904-ca40a9127b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.600478 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b24f3-6e8c-43b9-a904-ca40a9127b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.600755 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b24f3-6e8c-43b9-a904-ca40a9127b9b admin admin] PATH_INFO: `/v3/projects/4bea4e5dfb7b481c9f56a9104c915284/users/c039a7df792547a8b132261c1f6e97fc/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.615165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b24f3-6e8c-43b9-a904-ca40a9127b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bea4e5dfb7b481c9f56a9104c915284, user_id=c039a7df792547a8b132261c1f6e97fc, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:19.617844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b24f3-6e8c-43b9-a904-ca40a9127b9b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:19.631197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1348/2706] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:19 2022] PUT /identity/v3/projects/4bea4e5dfb7b481c9f56a9104c915284/users/c039a7df792547a8b132261c1f6e97fc/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:19.643128 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423bb32-331d-445a-8651-c987b54847e1 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.643410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423bb32-331d-445a-8651-c987b54847e1 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.643678 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423bb32-331d-445a-8651-c987b54847e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.670741 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b423bb32-331d-445a-8651-c987b54847e1 None None] MFA Rules not processed for user `c039a7df792547a8b132261c1f6e97fc`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:19.671924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b423bb32-331d-445a-8651-c987b54847e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:19.733886 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1359/2707] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:19 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:19.746177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b24f3-6e8c-43b9-a904-ca40a9127b9b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:19.747906 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-389b24f3-6e8c-43b9-a904-ca40a9127b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:19.770296 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:19.794212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423bb32-331d-445a-8651-c987b54847e1 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:19.795592 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b423bb32-331d-445a-8651-c987b54847e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:19.810273 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b24f3-6e8c-43b9-a904-ca40a9127b9b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:19.816288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87125d-b6a1-4e1c-a915-85909eff309c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:19.816920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87125d-b6a1-4e1c-a915-85909eff309c service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.817199 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87125d-b6a1-4e1c-a915-85909eff309c service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.817719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:19.817876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87125d-b6a1-4e1c-a915-85909eff309c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.827611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87125d-b6a1-4e1c-a915-85909eff309c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:19.829045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87125d-b6a1-4e1c-a915-85909eff309c service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:19.838825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1349/2708] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:19 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:19.866358 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423bb32-331d-445a-8651-c987b54847e1 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:19.871075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:19.871704 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.871978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.872243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] PATH_INFO: `/v3/users/a94b26aabfb3444d97dd5909515e0528` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.873367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a94b26aabfb3444d97dd5909515e0528)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:19.875337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:19.931628 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a94b26aabfb3444d97dd5909515e0528'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.939332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a94b26aabfb3444d97dd5909515e0528'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.940119 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a94b26aabfb3444d97dd5909515e0528'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.940908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a94b26aabfb3444d97dd5909515e0528'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.948484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a94b26aabfb3444d97dd5909515e0528'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.969789 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1360/2709] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:19 2022] DELETE /identity/v3/users/a94b26aabfb3444d97dd5909515e0528 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:19.973228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6758cf-3c65-4343-be2c-4ecae664c9da None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.973514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6758cf-3c65-4343-be2c-4ecae664c9da None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.973824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6758cf-3c65-4343-be2c-4ecae664c9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.994571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-bd6758cf-3c65-4343-be2c-4ecae664c9da None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:19.995782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bd6758cf-3c65-4343-be2c-4ecae664c9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:20.057772 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1350/2710] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:20.066993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:20.068378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:20.129207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b122a69-18bf-491f-a587-55f883b1c336 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:20.133561 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da64445-7976-45fe-a1d3-e2fe4e3c8232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:20.134202 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da64445-7976-45fe-a1d3-e2fe4e3c8232 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:20.135192 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da64445-7976-45fe-a1d3-e2fe4e3c8232 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:20.135371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da64445-7976-45fe-a1d3-e2fe4e3c8232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:20.144535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da64445-7976-45fe-a1d3-e2fe4e3c8232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:20.146024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da64445-7976-45fe-a1d3-e2fe4e3c8232 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:20.156507 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1361/2711] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:20.323383 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6758cf-3c65-4343-be2c-4ecae664c9da None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:20.324789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bd6758cf-3c65-4343-be2c-4ecae664c9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:20.339534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6758cf-3c65-4343-be2c-4ecae664c9da None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:20.343901 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991462a-7709-4b2d-b2e8-c6bdc5a940cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:20.344637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991462a-7709-4b2d-b2e8-c6bdc5a940cb service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:20.345120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991462a-7709-4b2d-b2e8-c6bdc5a940cb service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:20.345455 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991462a-7709-4b2d-b2e8-c6bdc5a940cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:20.347614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b991462a-7709-4b2d-b2e8-c6bdc5a940cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:20.362208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991462a-7709-4b2d-b2e8-c6bdc5a940cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:20.364096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991462a-7709-4b2d-b2e8-c6bdc5a940cb service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:20.378197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1351/2712] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:20.635344 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da64445-7976-45fe-a1d3-e2fe4e3c8232 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:20.636924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7da64445-7976-45fe-a1d3-e2fe4e3c8232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:20.650343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da64445-7976-45fe-a1d3-e2fe4e3c8232 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:20.655004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:20.655623 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:20.655905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:20.656181 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] PATH_INFO: `/v3/projects/633dfbdc2c4744f4aa582babff5de160` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:20.657350 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=633dfbdc2c4744f4aa582babff5de160)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:20.659410 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:20.710338 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:20.724057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '633dfbdc2c4744f4aa582babff5de160', 'request_id': 'req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:20.731360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '633dfbdc2c4744f4aa582babff5de160', 'request_id': 'req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:20.738712 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '633dfbdc2c4744f4aa582babff5de160', 'request_id': 'req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:20.750895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] The token cache is being invalidate because project 633dfbdc2c4744f4aa582babff5de160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:20.751183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:20.751819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:20.753299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1362/2713] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:20 2022] DELETE /identity/v3/projects/633dfbdc2c4744f4aa582babff5de160 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:20.757351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991462a-7709-4b2d-b2e8-c6bdc5a940cb service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:20.767332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991462a-7709-4b2d-b2e8-c6bdc5a940cb service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:20.824819 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce22824-21ec-4b7f-9fd5-e7b38e700fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:20.825780 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce22824-21ec-4b7f-9fd5-e7b38e700fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:20.826058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce22824-21ec-4b7f-9fd5-e7b38e700fef admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:20.826326 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce22824-21ec-4b7f-9fd5-e7b38e700fef admin admin] PATH_INFO: `/v3/users/24fffdc597d9448fab597aba7e9104e9` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:20.827618 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce22824-21ec-4b7f-9fd5-e7b38e700fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=24fffdc597d9448fab597aba7e9104e9)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:20.829527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce22824-21ec-4b7f-9fd5-e7b38e700fef admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:20.875940 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ce22824-21ec-4b7f-9fd5-e7b38e700fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24fffdc597d9448fab597aba7e9104e9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:20.879508 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:20.884165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ce22824-21ec-4b7f-9fd5-e7b38e700fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24fffdc597d9448fab597aba7e9104e9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:20.885442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ce22824-21ec-4b7f-9fd5-e7b38e700fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24fffdc597d9448fab597aba7e9104e9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:20.886231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ce22824-21ec-4b7f-9fd5-e7b38e700fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24fffdc597d9448fab597aba7e9104e9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:20.894420 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-8ce22824-21ec-4b7f-9fd5-e7b38e700fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24fffdc597d9448fab597aba7e9104e9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:20.921074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1352/2714] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:20 2022] DELETE /identity/v3/users/24fffdc597d9448fab597aba7e9104e9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:21.206156 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:21.208123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:21.222341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b1cd9-f8ba-49b2-b745-f84a49279c4c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:21.284544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:21.285093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:21.285371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:21.285640 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3 admin admin] PATH_INFO: `/v3/projects/9139c02f02c646e4b042b90b545bf003` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:21.286882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9139c02f02c646e4b042b90b545bf003)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:21.288783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:21.346209 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9139c02f02c646e4b042b90b545bf003', 'request_id': 'req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.375325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9139c02f02c646e4b042b90b545bf003', 'request_id': 'req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.386427 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9139c02f02c646e4b042b90b545bf003', 'request_id': 'req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.399061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3 admin admin] The token cache is being invalidate because project 9139c02f02c646e4b042b90b545bf003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:21.399389 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.401534 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-b491f4a6-c57a-46f9-bb35-0f2db981dfd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.403833 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1363/2715] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:21 2022] DELETE /identity/v3/projects/9139c02f02c646e4b042b90b545bf003 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:21.410847 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b129d1-093c-47db-b739-25b7a0d0a583 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:21.411335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b129d1-093c-47db-b739-25b7a0d0a583 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:21.412562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b129d1-093c-47db-b739-25b7a0d0a583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:21.436915 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f7b129d1-093c-47db-b739-25b7a0d0a583 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:21.438464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f7b129d1-093c-47db-b739-25b7a0d0a583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:21.529774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1353/2716] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:21.534908 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7e997-be2b-48c5-be1e-3474ee6ec6e3 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:21.535265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7e997-be2b-48c5-be1e-3474ee6ec6e3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:21.535994 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7e997-be2b-48c5-be1e-3474ee6ec6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:21.599570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-9ca7e997-be2b-48c5-be1e-3474ee6ec6e3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:21.601499 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-9ca7e997-be2b-48c5-be1e-3474ee6ec6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:21.611511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1364/2717] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:21.615751 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b129d1-093c-47db-b739-25b7a0d0a583 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:21.628823 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b129d1-093c-47db-b739-25b7a0d0a583 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:21.633497 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547435b-eb43-4694-962d-5ac43038b09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:21.634266 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547435b-eb43-4694-962d-5ac43038b09e admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:21.635077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547435b-eb43-4694-962d-5ac43038b09e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:21.635349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547435b-eb43-4694-962d-5ac43038b09e admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:21.636048 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547435b-eb43-4694-962d-5ac43038b09e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:21.636427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547435b-eb43-4694-962d-5ac43038b09e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:21.638092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547435b-eb43-4694-962d-5ac43038b09e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:21.650957 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1354/2718] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:02:21.655325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7e997-be2b-48c5-be1e-3474ee6ec6e3 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:21.668940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7e997-be2b-48c5-be1e-3474ee6ec6e3 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:21.674949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9ebb1-5cfd-493c-b178-c35999c26609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:21.676127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9ebb1-5cfd-493c-b178-c35999c26609 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:21.676616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9ebb1-5cfd-493c-b178-c35999c26609 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:21.676957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9ebb1-5cfd-493c-b178-c35999c26609 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:21.677674 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9ebb1-5cfd-493c-b178-c35999c26609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:21.679621 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d9ebb1-5cfd-493c-b178-c35999c26609 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:21.719002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1365/2719] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:21 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:21.723276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547435b-eb43-4694-962d-5ac43038b09e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:21.733505 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547435b-eb43-4694-962d-5ac43038b09e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:21.796545 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5490240-1f61-4ead-8760-0b474d74cd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:21.797154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5490240-1f61-4ead-8760-0b474d74cd63 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:21.797382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5490240-1f61-4ead-8760-0b474d74cd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:21.797651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5490240-1f61-4ead-8760-0b474d74cd63 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:21.798314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5490240-1f61-4ead-8760-0b474d74cd63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:21.800250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5490240-1f61-4ead-8760-0b474d74cd63 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:21.830690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1355/2720] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:21 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:21.835272 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9ebb1-5cfd-493c-b178-c35999c26609 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:21.848094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d9ebb1-5cfd-493c-b178-c35999c26609 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:21.853909 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ed9e1-7341-40c0-a29d-3ec5d6606d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:21.854657 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ed9e1-7341-40c0-a29d-3ec5d6606d41 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:21.855031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ed9e1-7341-40c0-a29d-3ec5d6606d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:21.855392 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ed9e1-7341-40c0-a29d-3ec5d6606d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:21.856274 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ed9e1-7341-40c0-a29d-3ec5d6606d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:21.857977 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ed9e1-7341-40c0-a29d-3ec5d6606d41 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:21.869815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1366/2721] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:21.874067 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5490240-1f61-4ead-8760-0b474d74cd63 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:21.884559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5490240-1f61-4ead-8760-0b474d74cd63 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:21.889344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc46cb-afb3-4c75-a8d6-cef56b9ea561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:21.889991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc46cb-afb3-4c75-a8d6-cef56b9ea561 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:21.890276 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc46cb-afb3-4c75-a8d6-cef56b9ea561 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:21.890549 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc46cb-afb3-4c75-a8d6-cef56b9ea561 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:21.891198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc46cb-afb3-4c75-a8d6-cef56b9ea561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:21.892838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc46cb-afb3-4c75-a8d6-cef56b9ea561 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:21.910384 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ed9e1-7341-40c0-a29d-3ec5d6606d41 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:21.915310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1356/2722] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:21.919666 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:21.925607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ed9e1-7341-40c0-a29d-3ec5d6606d41 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:21.934322 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09edbc18-66e7-482f-8265-5fb263b7e9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:21.935127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09edbc18-66e7-482f-8265-5fb263b7e9f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:21.935507 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09edbc18-66e7-482f-8265-5fb263b7e9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:21.935877 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09edbc18-66e7-482f-8265-5fb263b7e9f9 admin admin] PATH_INFO: `/v3/projects/57962b6b7b0541589abd69b82a5146e9/users/73e81b4b0a544cd4b9c8b16470b4f41b/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:21.952143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09edbc18-66e7-482f-8265-5fb263b7e9f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57962b6b7b0541589abd69b82a5146e9, user_id=73e81b4b0a544cd4b9c8b16470b4f41b, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:21.955589 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09edbc18-66e7-482f-8265-5fb263b7e9f9 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:21.974191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1367/2723] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:21 2022] PUT /identity/v3/projects/57962b6b7b0541589abd69b82a5146e9/users/73e81b4b0a544cd4b9c8b16470b4f41b/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:21.986160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05411dbf-a73f-4008-8dee-76fa5dbdb7c2 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:21.986454 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05411dbf-a73f-4008-8dee-76fa5dbdb7c2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:21.986732 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05411dbf-a73f-4008-8dee-76fa5dbdb7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:22.020740 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-05411dbf-a73f-4008-8dee-76fa5dbdb7c2 None None] MFA Rules not processed for user `73e81b4b0a544cd4b9c8b16470b4f41b`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:22.022139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-05411dbf-a73f-4008-8dee-76fa5dbdb7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:22.052867 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:22.104277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1357/2724] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:22.116817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09edbc18-66e7-482f-8265-5fb263b7e9f9 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:22.119254 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-09edbc18-66e7-482f-8265-5fb263b7e9f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:22.197360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09edbc18-66e7-482f-8265-5fb263b7e9f9 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:22.201141 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2d9f7-2392-4aa3-b985-e3cea5d70254 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e27f5b81c704dbebbd6a83b2713e237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:22.201877 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2d9f7-2392-4aa3-b985-e3cea5d70254 service glance] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:22.202157 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2d9f7-2392-4aa3-b985-e3cea5d70254 service glance] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:22.202426 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2d9f7-2392-4aa3-b985-e3cea5d70254 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:22.211810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2d9f7-2392-4aa3-b985-e3cea5d70254 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:22.213271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2d9f7-2392-4aa3-b985-e3cea5d70254 service glance] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:22.223357 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1368/2725] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:23.774657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05411dbf-a73f-4008-8dee-76fa5dbdb7c2 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:23.776483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-05411dbf-a73f-4008-8dee-76fa5dbdb7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:23.844305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05411dbf-a73f-4008-8dee-76fa5dbdb7c2 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:23.848801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ee9be-32c9-4bc1-a8c6-caf6ac4718b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:23.849432 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ee9be-32c9-4bc1-a8c6-caf6ac4718b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:23.849715 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ee9be-32c9-4bc1-a8c6-caf6ac4718b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:23.850026 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ee9be-32c9-4bc1-a8c6-caf6ac4718b5 admin admin] PATH_INFO: `/v3/users/75642df28f5d4301b979f39482a9a11d` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:23.851185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ee9be-32c9-4bc1-a8c6-caf6ac4718b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75642df28f5d4301b979f39482a9a11d)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:23.853137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ee9be-32c9-4bc1-a8c6-caf6ac4718b5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:23.902365 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-128ee9be-32c9-4bc1-a8c6-caf6ac4718b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75642df28f5d4301b979f39482a9a11d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.912480 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-128ee9be-32c9-4bc1-a8c6-caf6ac4718b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75642df28f5d4301b979f39482a9a11d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.913242 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-128ee9be-32c9-4bc1-a8c6-caf6ac4718b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75642df28f5d4301b979f39482a9a11d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.913887 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-128ee9be-32c9-4bc1-a8c6-caf6ac4718b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75642df28f5d4301b979f39482a9a11d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.921330 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-128ee9be-32c9-4bc1-a8c6-caf6ac4718b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75642df28f5d4301b979f39482a9a11d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.942293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1358/2726] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:23 2022] DELETE /identity/v3/users/75642df28f5d4301b979f39482a9a11d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:24.114036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2d9f7-2392-4aa3-b985-e3cea5d70254 service glance] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.127177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2d9f7-2392-4aa3-b985-e3cea5d70254 service glance] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.183563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03d49a-f479-4054-a712-3c58cdd99813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.184186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03d49a-f479-4054-a712-3c58cdd99813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.184483 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03d49a-f479-4054-a712-3c58cdd99813 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.184733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03d49a-f479-4054-a712-3c58cdd99813 admin admin] PATH_INFO: `/v3/projects/bc4476b1f4b84b5593ff2650592b9fb7` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.185867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03d49a-f479-4054-a712-3c58cdd99813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc4476b1f4b84b5593ff2650592b9fb7)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.187755 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03d49a-f479-4054-a712-3c58cdd99813 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.232789 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-db03d49a-f479-4054-a712-3c58cdd99813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc4476b1f4b84b5593ff2650592b9fb7', 'request_id': 'req-db03d49a-f479-4054-a712-3c58cdd99813'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:24.238938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-db03d49a-f479-4054-a712-3c58cdd99813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc4476b1f4b84b5593ff2650592b9fb7', 'request_id': 'req-db03d49a-f479-4054-a712-3c58cdd99813'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:24.247230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-db03d49a-f479-4054-a712-3c58cdd99813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc4476b1f4b84b5593ff2650592b9fb7', 'request_id': 'req-db03d49a-f479-4054-a712-3c58cdd99813'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:24.257095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-db03d49a-f479-4054-a712-3c58cdd99813 admin admin] The token cache is being invalidate because project bc4476b1f4b84b5593ff2650592b9fb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:24.257377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-db03d49a-f479-4054-a712-3c58cdd99813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:24.258017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-db03d49a-f479-4054-a712-3c58cdd99813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:24.260276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1369/2727] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:24 2022] DELETE /identity/v3/projects/bc4476b1f4b84b5593ff2650592b9fb7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:24.264900 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a2a51-1822-414e-aab2-50e9df1855ce None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.265191 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a2a51-1822-414e-aab2-50e9df1855ce None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.265467 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a2a51-1822-414e-aab2-50e9df1855ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.285631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-873a2a51-1822-414e-aab2-50e9df1855ce None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:24.286855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-873a2a51-1822-414e-aab2-50e9df1855ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:24.449979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:24.470520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1359/2728] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:24.478760 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e3099-b951-4bc1-b43a-a2121218f341 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.479073 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e3099-b951-4bc1-b43a-a2121218f341 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.479351 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e3099-b951-4bc1-b43a-a2121218f341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.499259 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-1b0e3099-b951-4bc1-b43a-a2121218f341 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:24.500458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-1b0e3099-b951-4bc1-b43a-a2121218f341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:24.508339 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1370/2729] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:24.512556 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a2a51-1822-414e-aab2-50e9df1855ce None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.522187 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a2a51-1822-414e-aab2-50e9df1855ce None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.526730 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b7739-5516-41e7-b9b0-934d342ef726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.527321 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7739-5516-41e7-b9b0-934d342ef726 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.527663 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7739-5516-41e7-b9b0-934d342ef726 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.527935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b7739-5516-41e7-b9b0-934d342ef726 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.528633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b7739-5516-41e7-b9b0-934d342ef726 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:24.529326 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b7739-5516-41e7-b9b0-934d342ef726 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.530526 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b7739-5516-41e7-b9b0-934d342ef726 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.542943 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1360/2730] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:02:24.547450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e3099-b951-4bc1-b43a-a2121218f341 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.557173 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e3099-b951-4bc1-b43a-a2121218f341 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.562320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f78192-2dd7-4cf0-8f74-738dafd2e0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.562926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f78192-2dd7-4cf0-8f74-738dafd2e0f2 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.563224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f78192-2dd7-4cf0-8f74-738dafd2e0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.563614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f78192-2dd7-4cf0-8f74-738dafd2e0f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.564257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f78192-2dd7-4cf0-8f74-738dafd2e0f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.566323 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f78192-2dd7-4cf0-8f74-738dafd2e0f2 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.571373 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:24.597830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1371/2731] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:24.602390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b7739-5516-41e7-b9b0-934d342ef726 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.614436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b7739-5516-41e7-b9b0-934d342ef726 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.671259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f7ae3-3e9a-4200-aa6e-4faaf9e6aea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.671801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f7ae3-3e9a-4200-aa6e-4faaf9e6aea5 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.672080 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f7ae3-3e9a-4200-aa6e-4faaf9e6aea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.672350 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f7ae3-3e9a-4200-aa6e-4faaf9e6aea5 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.672984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f7ae3-3e9a-4200-aa6e-4faaf9e6aea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.674920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f7ae3-3e9a-4200-aa6e-4faaf9e6aea5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.696753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1361/2732] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:24.701033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f78192-2dd7-4cf0-8f74-738dafd2e0f2 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.711243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f78192-2dd7-4cf0-8f74-738dafd2e0f2 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.716924 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db160cdb-5da5-42da-b9c7-b6539f3c2c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.717525 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db160cdb-5da5-42da-b9c7-b6539f3c2c50 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.717834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db160cdb-5da5-42da-b9c7-b6539f3c2c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.718108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db160cdb-5da5-42da-b9c7-b6539f3c2c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.718750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db160cdb-5da5-42da-b9c7-b6539f3c2c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.720400 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db160cdb-5da5-42da-b9c7-b6539f3c2c50 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.733796 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1372/2733] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:24.736552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f7ae3-3e9a-4200-aa6e-4faaf9e6aea5 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.750014 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f7ae3-3e9a-4200-aa6e-4faaf9e6aea5 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.760747 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566689d-2ede-4394-a419-6a4907b52697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.761387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566689d-2ede-4394-a419-6a4907b52697 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.761668 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566689d-2ede-4394-a419-6a4907b52697 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.762031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566689d-2ede-4394-a419-6a4907b52697 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.762676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566689d-2ede-4394-a419-6a4907b52697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.764369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566689d-2ede-4394-a419-6a4907b52697 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.777066 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1362/2734] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:24.781661 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db160cdb-5da5-42da-b9c7-b6539f3c2c50 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.793830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db160cdb-5da5-42da-b9c7-b6539f3c2c50 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.803026 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54decde3-baf8-4d8f-9a84-2466e2243c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.803701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54decde3-baf8-4d8f-9a84-2466e2243c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.803998 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54decde3-baf8-4d8f-9a84-2466e2243c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.804282 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54decde3-baf8-4d8f-9a84-2466e2243c05 admin admin] PATH_INFO: `/v3/projects/28678a9e22c448c999a9cd19ed9298ee/users/49ea4f4231914db18b99aa9f3e3c7b27/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.827698 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54decde3-baf8-4d8f-9a84-2466e2243c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28678a9e22c448c999a9cd19ed9298ee, user_id=49ea4f4231914db18b99aa9f3e3c7b27, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.831308 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54decde3-baf8-4d8f-9a84-2466e2243c05 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.848155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1373/2735] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:24 2022] PUT /identity/v3/projects/28678a9e22c448c999a9cd19ed9298ee/users/49ea4f4231914db18b99aa9f3e3c7b27/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:24.859253 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fce7f-be23-427e-9014-c344030f6c83 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.859536 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fce7f-be23-427e-9014-c344030f6c83 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.859808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fce7f-be23-427e-9014-c344030f6c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.887547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5d4fce7f-be23-427e-9014-c344030f6c83 None None] MFA Rules not processed for user `49ea4f4231914db18b99aa9f3e3c7b27`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:24.888736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5d4fce7f-be23-427e-9014-c344030f6c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:24.958625 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1363/2736] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:24.962897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54decde3-baf8-4d8f-9a84-2466e2243c05 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.027277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54decde3-baf8-4d8f-9a84-2466e2243c05 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.032616 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f65ecb-5dec-49ac-aaab-2c7693e29d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.033215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f65ecb-5dec-49ac-aaab-2c7693e29d4d admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.033557 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f65ecb-5dec-49ac-aaab-2c7693e29d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.033867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f65ecb-5dec-49ac-aaab-2c7693e29d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.034493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f65ecb-5dec-49ac-aaab-2c7693e29d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.036393 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f65ecb-5dec-49ac-aaab-2c7693e29d4d admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.064716 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1374/2737] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:25.068889 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4fce7f-be23-427e-9014-c344030f6c83 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.079934 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4fce7f-be23-427e-9014-c344030f6c83 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.117810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:25.137452 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eeb4ad-7d24-4595-bde6-4cc4f811f436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.138105 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eeb4ad-7d24-4595-bde6-4cc4f811f436 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.138395 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eeb4ad-7d24-4595-bde6-4cc4f811f436 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.138678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eeb4ad-7d24-4595-bde6-4cc4f811f436 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.139325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eeb4ad-7d24-4595-bde6-4cc4f811f436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.141233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eeb4ad-7d24-4595-bde6-4cc4f811f436 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.161083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1364/2738] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:25 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:25.165355 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f65ecb-5dec-49ac-aaab-2c7693e29d4d admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.176509 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f65ecb-5dec-49ac-aaab-2c7693e29d4d admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.181347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4aa84-487a-45d5-a69f-7ecbd13c8597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.182489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4aa84-487a-45d5-a69f-7ecbd13c8597 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.182757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4aa84-487a-45d5-a69f-7ecbd13c8597 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.183022 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4aa84-487a-45d5-a69f-7ecbd13c8597 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.183660 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4aa84-487a-45d5-a69f-7ecbd13c8597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.185233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e4aa84-487a-45d5-a69f-7ecbd13c8597 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.196057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1375/2739] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:25.200303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eeb4ad-7d24-4595-bde6-4cc4f811f436 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.210453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eeb4ad-7d24-4595-bde6-4cc4f811f436 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.214895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b4840-5df6-4fe9-b5af-0fc88d27c401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.215876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4840-5df6-4fe9-b5af-0fc88d27c401 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.216219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4840-5df6-4fe9-b5af-0fc88d27c401 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.216559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b4840-5df6-4fe9-b5af-0fc88d27c401 admin admin] PATH_INFO: `/v3/projects/3a5cf17705fc4d59a3e9387fa026ec33/users/6f6c28ac82124fafbda5f3cb914fe4ba/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.231337 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b4840-5df6-4fe9-b5af-0fc88d27c401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5cf17705fc4d59a3e9387fa026ec33, user_id=6f6c28ac82124fafbda5f3cb914fe4ba, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.234137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b4840-5df6-4fe9-b5af-0fc88d27c401 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.247653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1365/2740] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:25 2022] PUT /identity/v3/projects/3a5cf17705fc4d59a3e9387fa026ec33/users/6f6c28ac82124fafbda5f3cb914fe4ba/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:25.251841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4aa84-487a-45d5-a69f-7ecbd13c8597 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.261767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4aa84-487a-45d5-a69f-7ecbd13c8597 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.319803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f2ce2-630c-4c58-8bb9-424e4c41d387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.320449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f2ce2-630c-4c58-8bb9-424e4c41d387 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.320726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f2ce2-630c-4c58-8bb9-424e4c41d387 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.320996 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f2ce2-630c-4c58-8bb9-424e4c41d387 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.321669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f2ce2-630c-4c58-8bb9-424e4c41d387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.323343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f2ce2-630c-4c58-8bb9-424e4c41d387 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.333127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1376/2741] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:25.337484 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b4840-5df6-4fe9-b5af-0fc88d27c401 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.347955 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b4840-5df6-4fe9-b5af-0fc88d27c401 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.352599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebed13-01df-403c-81f9-080f63d54992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.353837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebed13-01df-403c-81f9-080f63d54992 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.354116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebed13-01df-403c-81f9-080f63d54992 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.354389 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebed13-01df-403c-81f9-080f63d54992 admin admin] PATH_INFO: `/v3/projects/3a5cf17705fc4d59a3e9387fa026ec33/users/6f6c28ac82124fafbda5f3cb914fe4ba/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.356321 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebed13-01df-403c-81f9-080f63d54992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5cf17705fc4d59a3e9387fa026ec33, user_id=6f6c28ac82124fafbda5f3cb914fe4ba, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.358995 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acebed13-01df-403c-81f9-080f63d54992 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.370301 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1366/2742] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:25 2022] PUT /identity/v3/projects/3a5cf17705fc4d59a3e9387fa026ec33/users/6f6c28ac82124fafbda5f3cb914fe4ba/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:25.384629 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc17e78-8c62-4d71-bcdd-4db5e24d2511 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.384978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc17e78-8c62-4d71-bcdd-4db5e24d2511 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.385304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc17e78-8c62-4d71-bcdd-4db5e24d2511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.414745 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-2bc17e78-8c62-4d71-bcdd-4db5e24d2511 None None] MFA Rules not processed for user `6f6c28ac82124fafbda5f3cb914fe4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:25.416521 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-2bc17e78-8c62-4d71-bcdd-4db5e24d2511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:25.425454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:25.487721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1377/2743] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:25.498786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebed13-01df-403c-81f9-080f63d54992 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.500487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-acebed13-01df-403c-81f9-080f63d54992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:25.558448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebed13-01df-403c-81f9-080f63d54992 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.562984 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf9a29-127c-4d4c-b98e-c7e70103c8e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.563681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf9a29-127c-4d4c-b98e-c7e70103c8e3 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.564039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf9a29-127c-4d4c-b98e-c7e70103c8e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.564325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf9a29-127c-4d4c-b98e-c7e70103c8e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.573555 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf9a29-127c-4d4c-b98e-c7e70103c8e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.575009 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf9a29-127c-4d4c-b98e-c7e70103c8e3 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.585314 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1367/2744] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:25.650505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc17e78-8c62-4d71-bcdd-4db5e24d2511 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.713644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc17e78-8c62-4d71-bcdd-4db5e24d2511 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.718676 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.719303 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.719583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.719855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] PATH_INFO: `/v3/users/49ea4f4231914db18b99aa9f3e3c7b27` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.721321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49ea4f4231914db18b99aa9f3e3c7b27)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.723525 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.771276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49ea4f4231914db18b99aa9f3e3c7b27'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.778800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49ea4f4231914db18b99aa9f3e3c7b27'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.779420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49ea4f4231914db18b99aa9f3e3c7b27'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.780040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49ea4f4231914db18b99aa9f3e3c7b27'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.787959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49ea4f4231914db18b99aa9f3e3c7b27'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.807176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1378/2745] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:25 2022] DELETE /identity/v3/users/49ea4f4231914db18b99aa9f3e3c7b27 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:25.811387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4af9c-f348-4ab0-81aa-3418761214e9 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.811791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4af9c-f348-4ab0-81aa-3418761214e9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.811940 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4af9c-f348-4ab0-81aa-3418761214e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.831877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-88d4af9c-f348-4ab0-81aa-3418761214e9 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:25.833074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-88d4af9c-f348-4ab0-81aa-3418761214e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:25.909758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1368/2746] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:25.923224 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.924788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:25.954789 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:25.989326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a3b98-4a53-4af3-99c0-0dad76601135 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.993720 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e9aac-756d-4cbb-975f-9c6ada663d79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.995076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e9aac-756d-4cbb-975f-9c6ada663d79 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.995352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e9aac-756d-4cbb-975f-9c6ada663d79 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.995631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e9aac-756d-4cbb-975f-9c6ada663d79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:26.006708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e9aac-756d-4cbb-975f-9c6ada663d79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:26.008173 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e9aac-756d-4cbb-975f-9c6ada663d79 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:26.018040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1379/2747] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:26.282731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4af9c-f348-4ab0-81aa-3418761214e9 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:26.284264 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-88d4af9c-f348-4ab0-81aa-3418761214e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:26.299806 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4af9c-f348-4ab0-81aa-3418761214e9 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:26.304433 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:26.305064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:26.305344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:26.305617 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] PATH_INFO: `/v3/projects/28678a9e22c448c999a9cd19ed9298ee` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:26.307164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28678a9e22c448c999a9cd19ed9298ee)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:26.309277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:26.348133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:26.367046 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28678a9e22c448c999a9cd19ed9298ee', 'request_id': 'req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.375840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28678a9e22c448c999a9cd19ed9298ee', 'request_id': 'req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.382604 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28678a9e22c448c999a9cd19ed9298ee', 'request_id': 'req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.395355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] The token cache is being invalidate because project 28678a9e22c448c999a9cd19ed9298ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:26.395626 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.396247 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.397721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1369/2748] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:26 2022] DELETE /identity/v3/projects/28678a9e22c448c999a9cd19ed9298ee => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:26.402424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e9aac-756d-4cbb-975f-9c6ada663d79 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:26.412235 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e9aac-756d-4cbb-975f-9c6ada663d79 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:26.470489 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6aaae-f860-4fc4-a5fb-311206565d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:26.471152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6aaae-f860-4fc4-a5fb-311206565d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:26.471434 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6aaae-f860-4fc4-a5fb-311206565d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:26.471711 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6aaae-f860-4fc4-a5fb-311206565d6e admin admin] PATH_INFO: `/v3/users/6f6c28ac82124fafbda5f3cb914fe4ba` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:26.472867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6aaae-f860-4fc4-a5fb-311206565d6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f6c28ac82124fafbda5f3cb914fe4ba)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:26.474855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6aaae-f860-4fc4-a5fb-311206565d6e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:26.523243 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-58b6aaae-f860-4fc4-a5fb-311206565d6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f6c28ac82124fafbda5f3cb914fe4ba'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.531046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-58b6aaae-f860-4fc4-a5fb-311206565d6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f6c28ac82124fafbda5f3cb914fe4ba'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.531790 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-58b6aaae-f860-4fc4-a5fb-311206565d6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f6c28ac82124fafbda5f3cb914fe4ba'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.532428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-58b6aaae-f860-4fc4-a5fb-311206565d6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f6c28ac82124fafbda5f3cb914fe4ba'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.539414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-58b6aaae-f860-4fc4-a5fb-311206565d6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f6c28ac82124fafbda5f3cb914fe4ba'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.557070 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1380/2749] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:26 2022] DELETE /identity/v3/users/6f6c28ac82124fafbda5f3cb914fe4ba => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:26.833523 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:26.834988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:26.848883 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a807cffe-9abe-47c6-ab7b-0fbe1ec040f5 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:26.911516 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:26.912281 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:26.912563 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:26.912837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] PATH_INFO: `/v3/projects/3a5cf17705fc4d59a3e9387fa026ec33` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:26.914026 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a5cf17705fc4d59a3e9387fa026ec33)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:26.916277 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:26.973953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5cf17705fc4d59a3e9387fa026ec33', 'request_id': 'req-13c15985-c399-40bd-9dfc-4155f22717f8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.973953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5cf17705fc4d59a3e9387fa026ec33', 'request_id': 'req-13c15985-c399-40bd-9dfc-4155f22717f8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.979261 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5cf17705fc4d59a3e9387fa026ec33', 'request_id': 'req-13c15985-c399-40bd-9dfc-4155f22717f8'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.992573 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] The token cache is being invalidate because project 3a5cf17705fc4d59a3e9387fa026ec33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:26.992866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.993491 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.995579 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1370/2750] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:26 2022] DELETE /identity/v3/projects/3a5cf17705fc4d59a3e9387fa026ec33 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:27.001204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0901-e519-482f-9a0f-f50624e1434e None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.001488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0901-e519-482f-9a0f-f50624e1434e None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.001793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e0901-e519-482f-9a0f-f50624e1434e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.024746 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-ff2e0901-e519-482f-9a0f-f50624e1434e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:27.026207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ff2e0901-e519-482f-9a0f-f50624e1434e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:27.031465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.032844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:27.070981 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:27.089207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:27.092938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1381/2751] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:27.101471 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26dd7a-78d5-4eba-a383-57bd9322a967 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.101896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26dd7a-78d5-4eba-a383-57bd9322a967 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.102180 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26dd7a-78d5-4eba-a383-57bd9322a967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.107566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c15985-c399-40bd-9dfc-4155f22717f8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.112670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e9135-6989-472e-b299-3505d3db7153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.113304 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e9135-6989-472e-b299-3505d3db7153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.113586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e9135-6989-472e-b299-3505d3db7153 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.113894 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e9135-6989-472e-b299-3505d3db7153 admin admin] PATH_INFO: `/v3/users/c039a7df792547a8b132261c1f6e97fc` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.115035 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e9135-6989-472e-b299-3505d3db7153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c039a7df792547a8b132261c1f6e97fc)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.116950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e9135-6989-472e-b299-3505d3db7153 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.122836 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-dc26dd7a-78d5-4eba-a383-57bd9322a967 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:27.124041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-dc26dd7a-78d5-4eba-a383-57bd9322a967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:27.131998 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1382/2752] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:27.136869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26dd7a-78d5-4eba-a383-57bd9322a967 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.147475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26dd7a-78d5-4eba-a383-57bd9322a967 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.152352 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd1fbe-d0cd-4903-a576-cca832a41c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.153310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd1fbe-d0cd-4903-a576-cca832a41c8e admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.153310 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd1fbe-d0cd-4903-a576-cca832a41c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.153590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd1fbe-d0cd-4903-a576-cca832a41c8e admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.154155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd1fbe-d0cd-4903-a576-cca832a41c8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:27.154449 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd1fbe-d0cd-4903-a576-cca832a41c8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.156031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd1fbe-d0cd-4903-a576-cca832a41c8e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.168216 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-cd8e9135-6989-472e-b299-3505d3db7153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c039a7df792547a8b132261c1f6e97fc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:27.173703 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1383/2753] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:02:27.174962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd1fbe-d0cd-4903-a576-cca832a41c8e admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.176019 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-cd8e9135-6989-472e-b299-3505d3db7153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c039a7df792547a8b132261c1f6e97fc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:27.176837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-cd8e9135-6989-472e-b299-3505d3db7153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c039a7df792547a8b132261c1f6e97fc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:27.177470 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-cd8e9135-6989-472e-b299-3505d3db7153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c039a7df792547a8b132261c1f6e97fc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:27.185106 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-cd8e9135-6989-472e-b299-3505d3db7153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c039a7df792547a8b132261c1f6e97fc'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:27.185347 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd1fbe-d0cd-4903-a576-cca832a41c8e admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.190275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47328e03-2a86-4978-8d8d-2e016154f658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.190864 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47328e03-2a86-4978-8d8d-2e016154f658 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.191140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47328e03-2a86-4978-8d8d-2e016154f658 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.191409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47328e03-2a86-4978-8d8d-2e016154f658 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.192065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47328e03-2a86-4978-8d8d-2e016154f658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.193972 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47328e03-2a86-4978-8d8d-2e016154f658 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.202708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1371/2754] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:27 2022] DELETE /identity/v3/users/c039a7df792547a8b132261c1f6e97fc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:27.207038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e35662-5b66-4b86-8268-3cdf4181f109 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.207320 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e35662-5b66-4b86-8268-3cdf4181f109 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.207593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e35662-5b66-4b86-8268-3cdf4181f109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.225855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1384/2755] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:27 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:27.226716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-f0e35662-5b66-4b86-8268-3cdf4181f109 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:27.227941 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f0e35662-5b66-4b86-8268-3cdf4181f109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:27.230020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47328e03-2a86-4978-8d8d-2e016154f658 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.232017 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-47328e03-2a86-4978-8d8d-2e016154f658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:27.292955 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1372/2756] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:27.298875 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47328e03-2a86-4978-8d8d-2e016154f658 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.303249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e35662-5b66-4b86-8268-3cdf4181f109 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.303598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016b757-0ea2-4674-9a4d-4ef371c4bb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.304215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016b757-0ea2-4674-9a4d-4ef371c4bb14 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.304492 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016b757-0ea2-4674-9a4d-4ef371c4bb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.304756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f0e35662-5b66-4b86-8268-3cdf4181f109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:27.304959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a016b757-0ea2-4674-9a4d-4ef371c4bb14 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.305398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016b757-0ea2-4674-9a4d-4ef371c4bb14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.307944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a016b757-0ea2-4674-9a4d-4ef371c4bb14 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.329724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1385/2757] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:27 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:27.334005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016b757-0ea2-4674-9a4d-4ef371c4bb14 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.345122 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a016b757-0ea2-4674-9a4d-4ef371c4bb14 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.350787 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12f6e0-5115-4646-b441-5d7cbce2ffae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.351407 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12f6e0-5115-4646-b441-5d7cbce2ffae admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.351686 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12f6e0-5115-4646-b441-5d7cbce2ffae admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.351955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12f6e0-5115-4646-b441-5d7cbce2ffae admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.352602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12f6e0-5115-4646-b441-5d7cbce2ffae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.354773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12f6e0-5115-4646-b441-5d7cbce2ffae admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.364185 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e35662-5b66-4b86-8268-3cdf4181f109 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.366011 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1386/2758] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:27.367799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480becb5-21c7-48a0-9e1e-a7a2742faeac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.368429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480becb5-21c7-48a0-9e1e-a7a2742faeac service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.369289 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480becb5-21c7-48a0-9e1e-a7a2742faeac service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.369614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480becb5-21c7-48a0-9e1e-a7a2742faeac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.370722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12f6e0-5115-4646-b441-5d7cbce2ffae admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.378867 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480becb5-21c7-48a0-9e1e-a7a2742faeac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.380303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480becb5-21c7-48a0-9e1e-a7a2742faeac service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.381135 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12f6e0-5115-4646-b441-5d7cbce2ffae admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.386167 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaeb51d-0056-4d6b-87a8-41b0fb193dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.386773 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaeb51d-0056-4d6b-87a8-41b0fb193dbe admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.387050 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaeb51d-0056-4d6b-87a8-41b0fb193dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.387321 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaeb51d-0056-4d6b-87a8-41b0fb193dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.387963 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaeb51d-0056-4d6b-87a8-41b0fb193dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.389535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaeb51d-0056-4d6b-87a8-41b0fb193dbe admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.391107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1373/2759] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:27.403598 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1387/2760] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:27.407824 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480becb5-21c7-48a0-9e1e-a7a2742faeac service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.419162 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480becb5-21c7-48a0-9e1e-a7a2742faeac service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.424045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede0a66-ec65-4fec-a9c8-41fe4c77eca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.424711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede0a66-ec65-4fec-a9c8-41fe4c77eca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.424992 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede0a66-ec65-4fec-a9c8-41fe4c77eca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.425272 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede0a66-ec65-4fec-a9c8-41fe4c77eca2 admin admin] PATH_INFO: `/v3/projects/73758182fb8c4c509296ceee27868ae3/users/a8eb1ecf73bd46079b8091ee74a2547f/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.435010 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:27.442970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede0a66-ec65-4fec-a9c8-41fe4c77eca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73758182fb8c4c509296ceee27868ae3, user_id=a8eb1ecf73bd46079b8091ee74a2547f, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.446327 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede0a66-ec65-4fec-a9c8-41fe4c77eca2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.468556 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1374/2761] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:27 2022] PUT /identity/v3/projects/73758182fb8c4c509296ceee27868ae3/users/a8eb1ecf73bd46079b8091ee74a2547f/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:27.479886 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a5e66-4567-4ede-be5c-9abd806fcf34 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.480176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a5e66-4567-4ede-be5c-9abd806fcf34 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.480451 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a5e66-4567-4ede-be5c-9abd806fcf34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.511152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-170a5e66-4567-4ede-be5c-9abd806fcf34 None None] MFA Rules not processed for user `a8eb1ecf73bd46079b8091ee74a2547f`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:27.512395 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-170a5e66-4567-4ede-be5c-9abd806fcf34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:27.591995 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1388/2762] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:27.596727 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede0a66-ec65-4fec-a9c8-41fe4c77eca2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.664571 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede0a66-ec65-4fec-a9c8-41fe4c77eca2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.670231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a2fa8-ae9a-426b-bdc5-d14ec2e43538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.670960 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a2fa8-ae9a-426b-bdc5-d14ec2e43538 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.671239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a2fa8-ae9a-426b-bdc5-d14ec2e43538 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.671508 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a2fa8-ae9a-426b-bdc5-d14ec2e43538 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.672161 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a2fa8-ae9a-426b-bdc5-d14ec2e43538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.674090 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a2fa8-ae9a-426b-bdc5-d14ec2e43538 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.711468 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1375/2763] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:27 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:27.713507 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a5e66-4567-4ede-be5c-9abd806fcf34 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.720867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:27.728803 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a5e66-4567-4ede-be5c-9abd806fcf34 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.789539 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cfc7f-68fb-414a-b56e-10c65b599b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.790294 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cfc7f-68fb-414a-b56e-10c65b599b91 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.790625 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cfc7f-68fb-414a-b56e-10c65b599b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.790893 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2cfc7f-68fb-414a-b56e-10c65b599b91 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.792014 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2cfc7f-68fb-414a-b56e-10c65b599b91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.793939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2cfc7f-68fb-414a-b56e-10c65b599b91 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.821596 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1389/2764] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:27 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:27.822865 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a2fa8-ae9a-426b-bdc5-d14ec2e43538 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.834394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a2fa8-ae9a-426b-bdc5-d14ec2e43538 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.842877 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6175c0a-9c4e-42db-aee7-7f9439096611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.843517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6175c0a-9c4e-42db-aee7-7f9439096611 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.843800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6175c0a-9c4e-42db-aee7-7f9439096611 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.844077 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6175c0a-9c4e-42db-aee7-7f9439096611 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.845725 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6175c0a-9c4e-42db-aee7-7f9439096611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.847445 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6175c0a-9c4e-42db-aee7-7f9439096611 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.859063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1376/2765] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:27.863185 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cfc7f-68fb-414a-b56e-10c65b599b91 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.873208 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2cfc7f-68fb-414a-b56e-10c65b599b91 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.877894 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77289bfe-60ba-4896-b1bc-f48140dbdcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.878551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77289bfe-60ba-4896-b1bc-f48140dbdcec admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.878822 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77289bfe-60ba-4896-b1bc-f48140dbdcec admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.879093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77289bfe-60ba-4896-b1bc-f48140dbdcec admin admin] PATH_INFO: `/v3/projects/b173f23e0ee94057a02c113b1254971c/users/39e86a06cb904f54a6e38755a956df0e/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.895428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77289bfe-60ba-4896-b1bc-f48140dbdcec admin admin] RBAC: Authorizing `identity:create_grant(project_id=b173f23e0ee94057a02c113b1254971c, user_id=39e86a06cb904f54a6e38755a956df0e, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.898156 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77289bfe-60ba-4896-b1bc-f48140dbdcec admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.916519 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1390/2766] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:27 2022] PUT /identity/v3/projects/b173f23e0ee94057a02c113b1254971c/users/39e86a06cb904f54a6e38755a956df0e/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:27.920777 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6175c0a-9c4e-42db-aee7-7f9439096611 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.934712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6175c0a-9c4e-42db-aee7-7f9439096611 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.955733 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77289bfe-60ba-4896-b1bc-f48140dbdcec admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.958288 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-77289bfe-60ba-4896-b1bc-f48140dbdcec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:28.010517 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04f3a6-8e5c-4cfc-a421-6d0bb29189a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.011137 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04f3a6-8e5c-4cfc-a421-6d0bb29189a7 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.011417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04f3a6-8e5c-4cfc-a421-6d0bb29189a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.011691 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04f3a6-8e5c-4cfc-a421-6d0bb29189a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.012352 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c04f3a6-8e5c-4cfc-a421-6d0bb29189a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.014173 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c04f3a6-8e5c-4cfc-a421-6d0bb29189a7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.027149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1377/2767] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:28.031351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04f3a6-8e5c-4cfc-a421-6d0bb29189a7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.042207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04f3a6-8e5c-4cfc-a421-6d0bb29189a7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.048959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede6b42-ac30-49fc-ab1e-ceb2ebd9008c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.049603 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede6b42-ac30-49fc-ab1e-ceb2ebd9008c admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.049920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede6b42-ac30-49fc-ab1e-ceb2ebd9008c admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.050193 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede6b42-ac30-49fc-ab1e-ceb2ebd9008c admin admin] PATH_INFO: `/v3/projects/b173f23e0ee94057a02c113b1254971c/users/39e86a06cb904f54a6e38755a956df0e/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.050942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77289bfe-60ba-4896-b1bc-f48140dbdcec admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.052106 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede6b42-ac30-49fc-ab1e-ceb2ebd9008c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b173f23e0ee94057a02c113b1254971c, user_id=39e86a06cb904f54a6e38755a956df0e, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.054781 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede6b42-ac30-49fc-ab1e-ceb2ebd9008c admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.056734 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fea24-6f38-4435-b3f5-0447e47f36ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.057520 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fea24-6f38-4435-b3f5-0447e47f36ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.058058 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fea24-6f38-4435-b3f5-0447e47f36ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.058570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fea24-6f38-4435-b3f5-0447e47f36ab admin admin] PATH_INFO: `/v3/projects/4bea4e5dfb7b481c9f56a9104c915284` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.059562 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fea24-6f38-4435-b3f5-0447e47f36ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bea4e5dfb7b481c9f56a9104c915284)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.061806 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fea24-6f38-4435-b3f5-0447e47f36ab admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.066575 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1378/2768] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:28 2022] PUT /identity/v3/projects/b173f23e0ee94057a02c113b1254971c/users/39e86a06cb904f54a6e38755a956df0e/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:28.077471 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6608ad-48ef-44f5-a748-525fdf65db55 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.077798 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6608ad-48ef-44f5-a748-525fdf65db55 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.078063 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6608ad-48ef-44f5-a748-525fdf65db55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.109318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-bb6608ad-48ef-44f5-a748-525fdf65db55 None None] MFA Rules not processed for user `39e86a06cb904f54a6e38755a956df0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:28.110498 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bb6608ad-48ef-44f5-a748-525fdf65db55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:28.113404 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-985fea24-6f38-4435-b3f5-0447e47f36ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bea4e5dfb7b481c9f56a9104c915284', 'request_id': 'req-985fea24-6f38-4435-b3f5-0447e47f36ab'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.121866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-985fea24-6f38-4435-b3f5-0447e47f36ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bea4e5dfb7b481c9f56a9104c915284', 'request_id': 'req-985fea24-6f38-4435-b3f5-0447e47f36ab'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.133362 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-985fea24-6f38-4435-b3f5-0447e47f36ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bea4e5dfb7b481c9f56a9104c915284', 'request_id': 'req-985fea24-6f38-4435-b3f5-0447e47f36ab'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.141587 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:28.148334 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-985fea24-6f38-4435-b3f5-0447e47f36ab admin admin] The token cache is being invalidate because project 4bea4e5dfb7b481c9f56a9104c915284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:28.148657 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:28.149044 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-985fea24-6f38-4435-b3f5-0447e47f36ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.151130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-985fea24-6f38-4435-b3f5-0447e47f36ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.153177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1391/2769] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:27 2022] DELETE /identity/v3/projects/4bea4e5dfb7b481c9f56a9104c915284 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:28.158780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9562b-29b6-4f67-a17b-332bfcbb5f7b None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.158780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9562b-29b6-4f67-a17b-332bfcbb5f7b None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.159012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9562b-29b6-4f67-a17b-332bfcbb5f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.179433 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-72b9562b-29b6-4f67-a17b-332bfcbb5f7b None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:28.180608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-72b9562b-29b6-4f67-a17b-332bfcbb5f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:28.187935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1379/2770] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:28.201204 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6608ad-48ef-44f5-a748-525fdf65db55 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.202639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-bb6608ad-48ef-44f5-a748-525fdf65db55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:28.246634 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1392/2771] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:28.255093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7f9d1-70ec-42aa-9349-8f6d890c22b3 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.255398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7f9d1-70ec-42aa-9349-8f6d890c22b3 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.258649 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7f9d1-70ec-42aa-9349-8f6d890c22b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.261472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6608ad-48ef-44f5-a748-525fdf65db55 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.265096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27693a-829c-4e0c-b7f8-452e7d7f021e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.265708 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27693a-829c-4e0c-b7f8-452e7d7f021e service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.266025 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27693a-829c-4e0c-b7f8-452e7d7f021e service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.266298 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc27693a-829c-4e0c-b7f8-452e7d7f021e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.275639 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27693a-829c-4e0c-b7f8-452e7d7f021e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.277061 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc27693a-829c-4e0c-b7f8-452e7d7f021e service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.278063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-a6c7f9d1-70ec-42aa-9349-8f6d890c22b3 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:28.279258 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-a6c7f9d1-70ec-42aa-9349-8f6d890c22b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:28.286387 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1380/2772] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:28.287204 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1393/2773] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:28.292585 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27693a-829c-4e0c-b7f8-452e7d7f021e service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.307370 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc27693a-829c-4e0c-b7f8-452e7d7f021e service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.312059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd4e0d-f320-414b-ae07-ba6fd603fb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.312904 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd4e0d-f320-414b-ae07-ba6fd603fb29 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.313188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd4e0d-f320-414b-ae07-ba6fd603fb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.313463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd4e0d-f320-414b-ae07-ba6fd603fb29 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.314145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd4e0d-f320-414b-ae07-ba6fd603fb29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:28.314453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd4e0d-f320-414b-ae07-ba6fd603fb29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.316083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd4e0d-f320-414b-ae07-ba6fd603fb29 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.333369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1381/2774] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:28.334748 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7f9d1-70ec-42aa-9349-8f6d890c22b3 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.344463 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7f9d1-70ec-42aa-9349-8f6d890c22b3 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.350800 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbda8b-8283-4a74-9beb-22d362766dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.350944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbda8b-8283-4a74-9beb-22d362766dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.351214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbda8b-8283-4a74-9beb-22d362766dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.351608 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbda8b-8283-4a74-9beb-22d362766dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.352279 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbda8b-8283-4a74-9beb-22d362766dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.356190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fbda8b-8283-4a74-9beb-22d362766dc3 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.385837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1394/2775] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:28 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:28.389787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd4e0d-f320-414b-ae07-ba6fd603fb29 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.400905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd4e0d-f320-414b-ae07-ba6fd603fb29 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.433002 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbda8b-8283-4a74-9beb-22d362766dc3 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.435203 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b9fbda8b-8283-4a74-9beb-22d362766dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:28.470862 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fd317-98ee-4bc1-959f-52ac6dddac80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.471466 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fd317-98ee-4bc1-959f-52ac6dddac80 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.471744 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fd317-98ee-4bc1-959f-52ac6dddac80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.472017 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fd317-98ee-4bc1-959f-52ac6dddac80 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.472678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fd317-98ee-4bc1-959f-52ac6dddac80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.474736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fd317-98ee-4bc1-959f-52ac6dddac80 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.499131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1382/2776] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:28 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:28.503453 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fd317-98ee-4bc1-959f-52ac6dddac80 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.506215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbda8b-8283-4a74-9beb-22d362766dc3 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.511504 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd977e85-c569-45f6-97f9-ba363f89c8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.512318 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977e85-c569-45f6-97f9-ba363f89c8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.512761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977e85-c569-45f6-97f9-ba363f89c8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.513297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977e85-c569-45f6-97f9-ba363f89c8e0 admin admin] PATH_INFO: `/v3/users/a8eb1ecf73bd46079b8091ee74a2547f` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.514032 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fd317-98ee-4bc1-959f-52ac6dddac80 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.514646 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd977e85-c569-45f6-97f9-ba363f89c8e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8eb1ecf73bd46079b8091ee74a2547f)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.516780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd977e85-c569-45f6-97f9-ba363f89c8e0 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.519789 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b560c-887b-494a-864a-59bf64ac4d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.520360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b560c-887b-494a-864a-59bf64ac4d1e admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.520759 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b560c-887b-494a-864a-59bf64ac4d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.521039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b560c-887b-494a-864a-59bf64ac4d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.521689 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b560c-887b-494a-864a-59bf64ac4d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.523354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b560c-887b-494a-864a-59bf64ac4d1e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.534678 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1383/2777] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:28.538646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b560c-887b-494a-864a-59bf64ac4d1e admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.551587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b560c-887b-494a-864a-59bf64ac4d1e admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.557097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f0c98-954e-4d30-ae7c-1376fe4c8260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.557741 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f0c98-954e-4d30-ae7c-1376fe4c8260 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.558033 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f0c98-954e-4d30-ae7c-1376fe4c8260 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.558309 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90f0c98-954e-4d30-ae7c-1376fe4c8260 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.559138 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f0c98-954e-4d30-ae7c-1376fe4c8260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.561146 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90f0c98-954e-4d30-ae7c-1376fe4c8260 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.572245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1384/2778] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:28.572445 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-fd977e85-c569-45f6-97f9-ba363f89c8e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8eb1ecf73bd46079b8091ee74a2547f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.575961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f0c98-954e-4d30-ae7c-1376fe4c8260 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.580547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-fd977e85-c569-45f6-97f9-ba363f89c8e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8eb1ecf73bd46079b8091ee74a2547f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.581214 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-fd977e85-c569-45f6-97f9-ba363f89c8e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8eb1ecf73bd46079b8091ee74a2547f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.581916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-fd977e85-c569-45f6-97f9-ba363f89c8e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8eb1ecf73bd46079b8091ee74a2547f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.586501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90f0c98-954e-4d30-ae7c-1376fe4c8260 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.589682 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-fd977e85-c569-45f6-97f9-ba363f89c8e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8eb1ecf73bd46079b8091ee74a2547f'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.592457 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366bdd57-de4e-4fda-b179-c66ec52b41a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.593133 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bdd57-de4e-4fda-b179-c66ec52b41a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.593418 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bdd57-de4e-4fda-b179-c66ec52b41a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.593882 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366bdd57-de4e-4fda-b179-c66ec52b41a1 admin admin] PATH_INFO: `/v3/projects/1dcae1b090af440f836b18a7bb000c27/users/39393f5448ed4c489ae1459d251f50d0/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.597690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:28.611068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1395/2779] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:28 2022] DELETE /identity/v3/users/a8eb1ecf73bd46079b8091ee74a2547f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:28.612103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366bdd57-de4e-4fda-b179-c66ec52b41a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dcae1b090af440f836b18a7bb000c27, user_id=39393f5448ed4c489ae1459d251f50d0, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.614831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366bdd57-de4e-4fda-b179-c66ec52b41a1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.615301 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde173ac-1eff-4500-812e-f748ae6787d0 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.615584 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde173ac-1eff-4500-812e-f748ae6787d0 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.615855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde173ac-1eff-4500-812e-f748ae6787d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.628155 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1385/2780] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:28 2022] PUT /identity/v3/projects/1dcae1b090af440f836b18a7bb000c27/users/39393f5448ed4c489ae1459d251f50d0/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:28.633413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246b89a-5a95-4b14-aff8-79ae9617461d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.633829 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246b89a-5a95-4b14-aff8-79ae9617461d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.634197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246b89a-5a95-4b14-aff8-79ae9617461d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.636181 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-bde173ac-1eff-4500-812e-f748ae6787d0 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:28.637398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-bde173ac-1eff-4500-812e-f748ae6787d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:28.641378 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:28.655840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6246b89a-5a95-4b14-aff8-79ae9617461d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:28.657045 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6246b89a-5a95-4b14-aff8-79ae9617461d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:28.703332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1396/2781] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:28.713683 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde173ac-1eff-4500-812e-f748ae6787d0 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.715445 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-bde173ac-1eff-4500-812e-f748ae6787d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:28.726586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1386/2782] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:28.741178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246b89a-5a95-4b14-aff8-79ae9617461d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.742753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6246b89a-5a95-4b14-aff8-79ae9617461d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:28.784867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde173ac-1eff-4500-812e-f748ae6787d0 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.788583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a16ca-4e7c-4695-9407-0370df45f8ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.789189 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a16ca-4e7c-4695-9407-0370df45f8ff service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.789469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a16ca-4e7c-4695-9407-0370df45f8ff service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.789837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a16ca-4e7c-4695-9407-0370df45f8ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.799437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a16ca-4e7c-4695-9407-0370df45f8ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.800905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a16ca-4e7c-4695-9407-0370df45f8ff service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.801609 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246b89a-5a95-4b14-aff8-79ae9617461d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.805267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3e0b1-9a87-4437-a6df-cad6ceca0593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.805926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3e0b1-9a87-4437-a6df-cad6ceca0593 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.806234 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3e0b1-9a87-4437-a6df-cad6ceca0593 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.806503 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3e0b1-9a87-4437-a6df-cad6ceca0593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.810681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1397/2783] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:28.819689 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3e0b1-9a87-4437-a6df-cad6ceca0593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.821139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3e0b1-9a87-4437-a6df-cad6ceca0593 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.831717 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1387/2784] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:29.333237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a16ca-4e7c-4695-9407-0370df45f8ff service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:29.334834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-639a16ca-4e7c-4695-9407-0370df45f8ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:29.349425 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a16ca-4e7c-4695-9407-0370df45f8ff service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:29.355086 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:29.355974 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:29.356316 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:29.356812 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] PATH_INFO: `/v3/projects/73758182fb8c4c509296ceee27868ae3` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:29.358713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] RBAC: Authorizing `identity:delete_project(project_id=73758182fb8c4c509296ceee27868ae3)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:29.360785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:29.416775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73758182fb8c4c509296ceee27868ae3', 'request_id': 'req-96769608-18df-4cd4-885f-d148815e84de'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.426631 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73758182fb8c4c509296ceee27868ae3', 'request_id': 'req-96769608-18df-4cd4-885f-d148815e84de'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.436307 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73758182fb8c4c509296ceee27868ae3', 'request_id': 'req-96769608-18df-4cd4-885f-d148815e84de'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.456839 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] The token cache is being invalidate because project 73758182fb8c4c509296ceee27868ae3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:29.457474 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.458359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.460072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1398/2785] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:29 2022] DELETE /identity/v3/projects/73758182fb8c4c509296ceee27868ae3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:29.464248 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3e0b1-9a87-4437-a6df-cad6ceca0593 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:29.477038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3e0b1-9a87-4437-a6df-cad6ceca0593 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:29.541161 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ab30b-bb43-4125-a255-bbe14e222170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:29.541931 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ab30b-bb43-4125-a255-bbe14e222170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:29.542222 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ab30b-bb43-4125-a255-bbe14e222170 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:29.542547 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ab30b-bb43-4125-a255-bbe14e222170 admin admin] PATH_INFO: `/v3/users/39e86a06cb904f54a6e38755a956df0e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:29.543753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ab30b-bb43-4125-a255-bbe14e222170 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e86a06cb904f54a6e38755a956df0e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:29.545722 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0ab30b-bb43-4125-a255-bbe14e222170 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:29.599267 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2d0ab30b-bb43-4125-a255-bbe14e222170 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e86a06cb904f54a6e38755a956df0e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.600903 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:29.608650 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2d0ab30b-bb43-4125-a255-bbe14e222170 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e86a06cb904f54a6e38755a956df0e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.609907 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2d0ab30b-bb43-4125-a255-bbe14e222170 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e86a06cb904f54a6e38755a956df0e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.610553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2d0ab30b-bb43-4125-a255-bbe14e222170 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e86a06cb904f54a6e38755a956df0e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.617868 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2d0ab30b-bb43-4125-a255-bbe14e222170 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e86a06cb904f54a6e38755a956df0e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.640160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1388/2786] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:29 2022] DELETE /identity/v3/users/39e86a06cb904f54a6e38755a956df0e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:29.950926 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:29.952377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:29.976930 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96769608-18df-4cd4-885f-d148815e84de admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:29.990897 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:30.047952 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70c6a2-5bec-47c5-952f-8284afca7ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:30.048821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70c6a2-5bec-47c5-952f-8284afca7ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.048971 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70c6a2-5bec-47c5-952f-8284afca7ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.049212 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70c6a2-5bec-47c5-952f-8284afca7ff1 admin admin] PATH_INFO: `/v3/projects/b173f23e0ee94057a02c113b1254971c` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.050548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70c6a2-5bec-47c5-952f-8284afca7ff1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b173f23e0ee94057a02c113b1254971c)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:30.052166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70c6a2-5bec-47c5-952f-8284afca7ff1 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:30.121711 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-af70c6a2-5bec-47c5-952f-8284afca7ff1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b173f23e0ee94057a02c113b1254971c', 'request_id': 'req-af70c6a2-5bec-47c5-952f-8284afca7ff1'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:30.130215 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-af70c6a2-5bec-47c5-952f-8284afca7ff1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b173f23e0ee94057a02c113b1254971c', 'request_id': 'req-af70c6a2-5bec-47c5-952f-8284afca7ff1'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:30.140034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-af70c6a2-5bec-47c5-952f-8284afca7ff1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b173f23e0ee94057a02c113b1254971c', 'request_id': 'req-af70c6a2-5bec-47c5-952f-8284afca7ff1'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:30.158461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-af70c6a2-5bec-47c5-952f-8284afca7ff1 admin admin] The token cache is being invalidate because project b173f23e0ee94057a02c113b1254971c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:30.159038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-af70c6a2-5bec-47c5-952f-8284afca7ff1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:30.159753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-af70c6a2-5bec-47c5-952f-8284afca7ff1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:30.162417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1399/2787] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:29 2022] DELETE /identity/v3/projects/b173f23e0ee94057a02c113b1254971c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:30.167706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1c0ea-c6ca-4b0c-a04c-dc90905e1513 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.168059 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1c0ea-c6ca-4b0c-a04c-dc90905e1513 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.168400 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1c0ea-c6ca-4b0c-a04c-dc90905e1513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.198593 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-d8e1c0ea-c6ca-4b0c-a04c-dc90905e1513 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:30.199886 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-d8e1c0ea-c6ca-4b0c-a04c-dc90905e1513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:30.283465 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1389/2788] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:30.292957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03b969-0983-4787-bb57-0cd476998872 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.293238 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03b969-0983-4787-bb57-0cd476998872 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.293512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03b969-0983-4787-bb57-0cd476998872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.318637 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-7a03b969-0983-4787-bb57-0cd476998872 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:30.319739 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7a03b969-0983-4787-bb57-0cd476998872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:30.333697 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1400/2789] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:30.334981 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1c0ea-c6ca-4b0c-a04c-dc90905e1513 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:30.345278 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1c0ea-c6ca-4b0c-a04c-dc90905e1513 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:30.350718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6deb5-0935-468e-b7d2-84be5cffc98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:30.351290 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6deb5-0935-468e-b7d2-84be5cffc98a admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.351565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6deb5-0935-468e-b7d2-84be5cffc98a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.351820 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6deb5-0935-468e-b7d2-84be5cffc98a admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.352464 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6deb5-0935-468e-b7d2-84be5cffc98a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:30.352763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6deb5-0935-468e-b7d2-84be5cffc98a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:30.354929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6deb5-0935-468e-b7d2-84be5cffc98a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:30.370394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1390/2790] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:30.375061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03b969-0983-4787-bb57-0cd476998872 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:30.384487 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03b969-0983-4787-bb57-0cd476998872 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:30.389579 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf35e3-70a1-419c-8057-504821a10208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:30.390614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf35e3-70a1-419c-8057-504821a10208 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.390614 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf35e3-70a1-419c-8057-504821a10208 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.390861 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf35e3-70a1-419c-8057-504821a10208 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.391365 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf35e3-70a1-419c-8057-504821a10208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:30.393367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf35e3-70a1-419c-8057-504821a10208 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:30.434818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1401/2791] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:30 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:30.435758 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6deb5-0935-468e-b7d2-84be5cffc98a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:30.446838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6deb5-0935-468e-b7d2-84be5cffc98a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:30.516429 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361790a0-a84f-4cfb-8fd6-a345455dfabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:30.517258 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361790a0-a84f-4cfb-8fd6-a345455dfabd admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.517447 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361790a0-a84f-4cfb-8fd6-a345455dfabd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.517680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361790a0-a84f-4cfb-8fd6-a345455dfabd admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.518365 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361790a0-a84f-4cfb-8fd6-a345455dfabd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:30.520542 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361790a0-a84f-4cfb-8fd6-a345455dfabd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:30.549779 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1391/2792] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:30 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:30.553669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf35e3-70a1-419c-8057-504821a10208 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:30.568540 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf35e3-70a1-419c-8057-504821a10208 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:30.574020 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa83a6-e7d1-4319-902e-327fd59580d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:30.574821 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa83a6-e7d1-4319-902e-327fd59580d5 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.575123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa83a6-e7d1-4319-902e-327fd59580d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.575442 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa83a6-e7d1-4319-902e-327fd59580d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.576102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa83a6-e7d1-4319-902e-327fd59580d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:30.578131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa83a6-e7d1-4319-902e-327fd59580d5 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:30.591424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1402/2793] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:30.595667 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361790a0-a84f-4cfb-8fd6-a345455dfabd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:30.607921 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361790a0-a84f-4cfb-8fd6-a345455dfabd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:30.612792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b1183-4fcc-4265-b85c-543ee7d1f00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:30.613414 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b1183-4fcc-4265-b85c-543ee7d1f00e admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.613695 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b1183-4fcc-4265-b85c-543ee7d1f00e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.614260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b1183-4fcc-4265-b85c-543ee7d1f00e admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.615072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b1183-4fcc-4265-b85c-543ee7d1f00e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:30.616857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297b1183-4fcc-4265-b85c-543ee7d1f00e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:30.633293 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1392/2794] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:30.635653 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa83a6-e7d1-4319-902e-327fd59580d5 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:30.646331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa83a6-e7d1-4319-902e-327fd59580d5 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:30.651883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5b576-28f9-48ee-9e3c-c1091a3e8901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:30.652536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5b576-28f9-48ee-9e3c-c1091a3e8901 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.652815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5b576-28f9-48ee-9e3c-c1091a3e8901 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.653085 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5b576-28f9-48ee-9e3c-c1091a3e8901 admin admin] PATH_INFO: `/v3/projects/1c841e6e36f64e6f828991790a45b7c7/users/323b51d8b91d45df9e77daf0b5f4417c/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.704090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5b576-28f9-48ee-9e3c-c1091a3e8901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c841e6e36f64e6f828991790a45b7c7, user_id=323b51d8b91d45df9e77daf0b5f4417c, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:30.706761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5b576-28f9-48ee-9e3c-c1091a3e8901 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:30.720600 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1403/2795] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:30 2022] PUT /identity/v3/projects/1c841e6e36f64e6f828991790a45b7c7/users/323b51d8b91d45df9e77daf0b5f4417c/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:30.735254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7887c47-49cc-480d-ac1d-f135313cd896 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.735254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7887c47-49cc-480d-ac1d-f135313cd896 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.735254 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7887c47-49cc-480d-ac1d-f135313cd896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.764586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-a7887c47-49cc-480d-ac1d-f135313cd896 None None] MFA Rules not processed for user `323b51d8b91d45df9e77daf0b5f4417c`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:30.765866 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-a7887c47-49cc-480d-ac1d-f135313cd896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:30.788908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:30.862348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1393/2796] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:30.874829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5b576-28f9-48ee-9e3c-c1091a3e8901 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:30.876688 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7ce5b576-28f9-48ee-9e3c-c1091a3e8901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:30.930698 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:30.944610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5b576-28f9-48ee-9e3c-c1091a3e8901 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:30.948605 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fd8b1-e05f-474c-b92f-21e7705c6822 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:30.949220 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fd8b1-e05f-474c-b92f-21e7705c6822 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.949505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fd8b1-e05f-474c-b92f-21e7705c6822 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.949802 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fd8b1-e05f-474c-b92f-21e7705c6822 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.958793 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fd8b1-e05f-474c-b92f-21e7705c6822 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:30.960241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fd8b1-e05f-474c-b92f-21e7705c6822 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:30.971112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1404/2797] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:33.229521 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ba291-7e59-4a4c-9fc1-895945fbca90 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.230111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ba291-7e59-4a4c-9fc1-895945fbca90 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.233143 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ba291-7e59-4a4c-9fc1-895945fbca90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.267407 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-143ba291-7e59-4a4c-9fc1-895945fbca90 None None] MFA Rules not processed for user `39393f5448ed4c489ae1459d251f50d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:33.268633 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-143ba291-7e59-4a4c-9fc1-895945fbca90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:33.342380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1394/2798] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:33.346663 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fd8b1-e05f-474c-b92f-21e7705c6822 service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.349572 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5e6fd8b1-e05f-474c-b92f-21e7705c6822 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:33.434249 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fd8b1-e05f-474c-b92f-21e7705c6822 service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.438914 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e17b42-b03c-4d3b-b985-58230a9c2084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.439599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e17b42-b03c-4d3b-b985-58230a9c2084 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.439949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e17b42-b03c-4d3b-b985-58230a9c2084 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.440312 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e17b42-b03c-4d3b-b985-58230a9c2084 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.441018 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e17b42-b03c-4d3b-b985-58230a9c2084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.443067 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e17b42-b03c-4d3b-b985-58230a9c2084 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.474409 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1405/2799] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:33 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:33.476081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ba291-7e59-4a4c-9fc1-895945fbca90 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.490032 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ba291-7e59-4a4c-9fc1-895945fbca90 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.551285 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261363ed-6ffe-4e43-959a-69cd92789f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.551908 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261363ed-6ffe-4e43-959a-69cd92789f93 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.552188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261363ed-6ffe-4e43-959a-69cd92789f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.552469 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261363ed-6ffe-4e43-959a-69cd92789f93 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.553104 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261363ed-6ffe-4e43-959a-69cd92789f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.555064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261363ed-6ffe-4e43-959a-69cd92789f93 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.579642 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1395/2800] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:33 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:33.583693 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e17b42-b03c-4d3b-b985-58230a9c2084 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.594760 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e17b42-b03c-4d3b-b985-58230a9c2084 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.600152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b6eb9-bb0d-4f3f-ac45-15586d900d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.600945 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b6eb9-bb0d-4f3f-ac45-15586d900d56 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.601287 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b6eb9-bb0d-4f3f-ac45-15586d900d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.601625 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b6eb9-bb0d-4f3f-ac45-15586d900d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.602385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b6eb9-bb0d-4f3f-ac45-15586d900d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.604563 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b6eb9-bb0d-4f3f-ac45-15586d900d56 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.616036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1406/2801] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:33.620646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261363ed-6ffe-4e43-959a-69cd92789f93 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.630875 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261363ed-6ffe-4e43-959a-69cd92789f93 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.635991 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75ced1-fcf6-4120-818a-2c59d0e281f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.636682 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75ced1-fcf6-4120-818a-2c59d0e281f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.636961 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75ced1-fcf6-4120-818a-2c59d0e281f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.637240 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75ced1-fcf6-4120-818a-2c59d0e281f7 admin admin] PATH_INFO: `/v3/projects/c242bca3eda94d1c9ae0121553a0f12f/users/a605f837a1004c1f98d8246af920543e/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.654259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75ced1-fcf6-4120-818a-2c59d0e281f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c242bca3eda94d1c9ae0121553a0f12f, user_id=a605f837a1004c1f98d8246af920543e, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.656935 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f75ced1-fcf6-4120-818a-2c59d0e281f7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.675898 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1396/2802] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:33 2022] PUT /identity/v3/projects/c242bca3eda94d1c9ae0121553a0f12f/users/a605f837a1004c1f98d8246af920543e/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:33.677074 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b6eb9-bb0d-4f3f-ac45-15586d900d56 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.688094 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b6eb9-bb0d-4f3f-ac45-15586d900d56 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.748836 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ec2f2-7973-4d3b-b337-cad8c924a9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.749354 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ec2f2-7973-4d3b-b337-cad8c924a9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.749632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ec2f2-7973-4d3b-b337-cad8c924a9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.749934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ec2f2-7973-4d3b-b337-cad8c924a9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.750582 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ec2f2-7973-4d3b-b337-cad8c924a9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.752366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ec2f2-7973-4d3b-b337-cad8c924a9d4 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.762848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1407/2803] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:33.766925 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75ced1-fcf6-4120-818a-2c59d0e281f7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.777949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75ced1-fcf6-4120-818a-2c59d0e281f7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.782559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afff1ab-02cf-40c2-8e73-f7ec51bff4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.783209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afff1ab-02cf-40c2-8e73-f7ec51bff4d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.783490 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afff1ab-02cf-40c2-8e73-f7ec51bff4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.783793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afff1ab-02cf-40c2-8e73-f7ec51bff4d0 admin admin] PATH_INFO: `/v3/projects/c242bca3eda94d1c9ae0121553a0f12f/users/a605f837a1004c1f98d8246af920543e/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.785694 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afff1ab-02cf-40c2-8e73-f7ec51bff4d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c242bca3eda94d1c9ae0121553a0f12f, user_id=a605f837a1004c1f98d8246af920543e, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.788414 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afff1ab-02cf-40c2-8e73-f7ec51bff4d0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.799156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1397/2804] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:33 2022] PUT /identity/v3/projects/c242bca3eda94d1c9ae0121553a0f12f/users/a605f837a1004c1f98d8246af920543e/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:38.746370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63893964-e32f-4495-b9f5-13d5ed17d12f None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:38.747388 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63893964-e32f-4495-b9f5-13d5ed17d12f None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:38.747849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63893964-e32f-4495-b9f5-13d5ed17d12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:38.793533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-63893964-e32f-4495-b9f5-13d5ed17d12f None None] MFA Rules not processed for user `a605f837a1004c1f98d8246af920543e`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:38.795439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-63893964-e32f-4495-b9f5-13d5ed17d12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:38.850031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:38.878547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1408/2805] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:38.893753 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afff1ab-02cf-40c2-8e73-f7ec51bff4d0 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:38.896147 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6afff1ab-02cf-40c2-8e73-f7ec51bff4d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:38.902852 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:38.980527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afff1ab-02cf-40c2-8e73-f7ec51bff4d0 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:38.986230 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8754a9-623c-4b70-9847-8332defdd27a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:38.986861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8754a9-623c-4b70-9847-8332defdd27a service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:38.987149 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8754a9-623c-4b70-9847-8332defdd27a service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:38.987424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8754a9-623c-4b70-9847-8332defdd27a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:39.046178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8754a9-623c-4b70-9847-8332defdd27a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:39.047782 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8754a9-623c-4b70-9847-8332defdd27a service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:39.057833 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1398/2806] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:40.075110 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44860c-2c48-48bf-befd-824e45bd4d0c None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.075719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44860c-2c48-48bf-befd-824e45bd4d0c None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.075719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44860c-2c48-48bf-befd-824e45bd4d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.094785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-ff44860c-2c48-48bf-befd-824e45bd4d0c None None] MFA Rules not processed for user `39393f5448ed4c489ae1459d251f50d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:40.096036 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-ff44860c-2c48-48bf-befd-824e45bd4d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:40.103673 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1409/2807] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:40.283934 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8754a9-623c-4b70-9847-8332defdd27a service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:40.365917 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8754a9-623c-4b70-9847-8332defdd27a service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:40.370460 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:40.371088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.371365 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.371643 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] PATH_INFO: `/v3/users/323b51d8b91d45df9e77daf0b5f4417c` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.372757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=323b51d8b91d45df9e77daf0b5f4417c)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:40.374817 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:40.421929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:40.429394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '323b51d8b91d45df9e77daf0b5f4417c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:40.439308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '323b51d8b91d45df9e77daf0b5f4417c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:40.440055 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '323b51d8b91d45df9e77daf0b5f4417c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:40.440812 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '323b51d8b91d45df9e77daf0b5f4417c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:40.450328 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '323b51d8b91d45df9e77daf0b5f4417c'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:40.475224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1399/2808] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:40 2022] DELETE /identity/v3/users/323b51d8b91d45df9e77daf0b5f4417c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:40.477198 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d0e53-5ebd-4d94-81be-852c9186eb57 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.477717 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d0e53-5ebd-4d94-81be-852c9186eb57 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.478039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d0e53-5ebd-4d94-81be-852c9186eb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.504755 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-6d2d0e53-5ebd-4d94-81be-852c9186eb57 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:40.506512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-6d2d0e53-5ebd-4d94-81be-852c9186eb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:40.601976 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1410/2809] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:40.616122 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:40.618165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:40.683348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220724c8-24d4-4524-a68c-dab1263dd3a6 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:40.687002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5632fe5-d801-45ec-8b79-d51684f34a78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:40.687686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5632fe5-d801-45ec-8b79-d51684f34a78 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.687969 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5632fe5-d801-45ec-8b79-d51684f34a78 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.688241 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5632fe5-d801-45ec-8b79-d51684f34a78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.697354 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5632fe5-d801-45ec-8b79-d51684f34a78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:40.698814 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5632fe5-d801-45ec-8b79-d51684f34a78 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:40.708392 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1400/2810] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:40.999420 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d0e53-5ebd-4d94-81be-852c9186eb57 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.001029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-6d2d0e53-5ebd-4d94-81be-852c9186eb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.018186 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d0e53-5ebd-4d94-81be-852c9186eb57 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.024367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90284da4-5c65-4418-8e17-bae5d1e942c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.025560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90284da4-5c65-4418-8e17-bae5d1e942c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.025896 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90284da4-5c65-4418-8e17-bae5d1e942c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.026170 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90284da4-5c65-4418-8e17-bae5d1e942c1 admin admin] PATH_INFO: `/v3/projects/1c841e6e36f64e6f828991790a45b7c7` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.027400 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90284da4-5c65-4418-8e17-bae5d1e942c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c841e6e36f64e6f828991790a45b7c7)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.029554 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90284da4-5c65-4418-8e17-bae5d1e942c1 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.087502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-90284da4-5c65-4418-8e17-bae5d1e942c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c841e6e36f64e6f828991790a45b7c7', 'request_id': 'req-90284da4-5c65-4418-8e17-bae5d1e942c1'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:41.095087 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-90284da4-5c65-4418-8e17-bae5d1e942c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c841e6e36f64e6f828991790a45b7c7', 'request_id': 'req-90284da4-5c65-4418-8e17-bae5d1e942c1'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:41.104665 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-90284da4-5c65-4418-8e17-bae5d1e942c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c841e6e36f64e6f828991790a45b7c7', 'request_id': 'req-90284da4-5c65-4418-8e17-bae5d1e942c1'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:41.115543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-90284da4-5c65-4418-8e17-bae5d1e942c1 admin admin] The token cache is being invalidate because project 1c841e6e36f64e6f828991790a45b7c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:41.115832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-90284da4-5c65-4418-8e17-bae5d1e942c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:41.116485 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-90284da4-5c65-4418-8e17-bae5d1e942c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:41.118037 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1411/2811] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:40 2022] DELETE /identity/v3/projects/1c841e6e36f64e6f828991790a45b7c7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:41.123656 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac29dc-62a8-4302-9225-c2f99d198ea1 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.124210 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac29dc-62a8-4302-9225-c2f99d198ea1 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.124553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac29dc-62a8-4302-9225-c2f99d198ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.145092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e0ac29dc-62a8-4302-9225-c2f99d198ea1 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:41.146412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e0ac29dc-62a8-4302-9225-c2f99d198ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.212499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1401/2812] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:41.220810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486cdd1-e1bc-4366-8971-7fae6131949b None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.221093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486cdd1-e1bc-4366-8971-7fae6131949b None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.221371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486cdd1-e1bc-4366-8971-7fae6131949b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.244034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-3486cdd1-e1bc-4366-8971-7fae6131949b None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:41.245260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-3486cdd1-e1bc-4366-8971-7fae6131949b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.253632 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1412/2813] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:41.257861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac29dc-62a8-4302-9225-c2f99d198ea1 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.268658 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac29dc-62a8-4302-9225-c2f99d198ea1 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.273318 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9376f-e6c3-4e6c-b43a-7246bc53d0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.273959 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9376f-e6c3-4e6c-b43a-7246bc53d0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.274239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9376f-e6c3-4e6c-b43a-7246bc53d0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.274509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9376f-e6c3-4e6c-b43a-7246bc53d0d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.275160 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9376f-e6c3-4e6c-b43a-7246bc53d0d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:41.275461 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9376f-e6c3-4e6c-b43a-7246bc53d0d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.277125 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9376f-e6c3-4e6c-b43a-7246bc53d0d2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.289609 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1402/2814] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:02:41.295662 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486cdd1-e1bc-4366-8971-7fae6131949b None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.305369 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486cdd1-e1bc-4366-8971-7fae6131949b None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.310687 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6e964-bc75-4e1c-b3f0-962347bafa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.311290 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6e964-bc75-4e1c-b3f0-962347bafa74 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.311565 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6e964-bc75-4e1c-b3f0-962347bafa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.311832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6e964-bc75-4e1c-b3f0-962347bafa74 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.312503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa6e964-bc75-4e1c-b3f0-962347bafa74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.314464 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa6e964-bc75-4e1c-b3f0-962347bafa74 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.342607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1413/2815] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:41 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:41.346853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9376f-e6c3-4e6c-b43a-7246bc53d0d2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.357615 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9376f-e6c3-4e6c-b43a-7246bc53d0d2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.409422 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:41.422565 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76709f5-2dbf-4758-92cd-a7a74a2bcef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.423184 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76709f5-2dbf-4758-92cd-a7a74a2bcef1 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.423462 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76709f5-2dbf-4758-92cd-a7a74a2bcef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.423835 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76709f5-2dbf-4758-92cd-a7a74a2bcef1 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.424485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76709f5-2dbf-4758-92cd-a7a74a2bcef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.426626 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76709f5-2dbf-4758-92cd-a7a74a2bcef1 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.451412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1403/2816] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:41 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:41.455481 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6e964-bc75-4e1c-b3f0-962347bafa74 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.466980 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6e964-bc75-4e1c-b3f0-962347bafa74 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.471722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f507f2-1b3a-4651-bcba-061c2937b636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.472327 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f507f2-1b3a-4651-bcba-061c2937b636 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.472602 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f507f2-1b3a-4651-bcba-061c2937b636 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.472868 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f507f2-1b3a-4651-bcba-061c2937b636 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.473514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f507f2-1b3a-4651-bcba-061c2937b636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.475140 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f507f2-1b3a-4651-bcba-061c2937b636 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.485464 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1414/2817] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:41.489872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76709f5-2dbf-4758-92cd-a7a74a2bcef1 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.501020 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76709f5-2dbf-4758-92cd-a7a74a2bcef1 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.505934 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc622f-5ee4-40f8-9138-de7fb55f6992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.506674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc622f-5ee4-40f8-9138-de7fb55f6992 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.506963 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc622f-5ee4-40f8-9138-de7fb55f6992 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.507351 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc622f-5ee4-40f8-9138-de7fb55f6992 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.508048 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc622f-5ee4-40f8-9138-de7fb55f6992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.510512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc622f-5ee4-40f8-9138-de7fb55f6992 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.522858 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1404/2818] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:41.527075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f507f2-1b3a-4651-bcba-061c2937b636 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.537246 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f507f2-1b3a-4651-bcba-061c2937b636 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.542797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be238f-a1b0-4d53-8a37-e3a2a2af73f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.543445 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be238f-a1b0-4d53-8a37-e3a2a2af73f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.546638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be238f-a1b0-4d53-8a37-e3a2a2af73f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.546952 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be238f-a1b0-4d53-8a37-e3a2a2af73f6 admin admin] PATH_INFO: `/v3/projects/6a848414c86b439c922442d0862c334f/users/52bda0133af44d5ca220ba869fbcb3a9/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.566544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be238f-a1b0-4d53-8a37-e3a2a2af73f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a848414c86b439c922442d0862c334f, user_id=52bda0133af44d5ca220ba869fbcb3a9, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.569286 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be238f-a1b0-4d53-8a37-e3a2a2af73f6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.571658 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:41.587929 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1415/2819] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:41 2022] PUT /identity/v3/projects/6a848414c86b439c922442d0862c334f/users/52bda0133af44d5ca220ba869fbcb3a9/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:41.593475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473a2e9-f304-490e-b260-f2b7afb6eecc None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.593475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473a2e9-f304-490e-b260-f2b7afb6eecc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.593816 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e473a2e9-f304-490e-b260-f2b7afb6eecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.616421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e473a2e9-f304-490e-b260-f2b7afb6eecc None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:41.617612 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e473a2e9-f304-490e-b260-f2b7afb6eecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.682752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1405/2820] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:41.693019 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be238f-a1b0-4d53-8a37-e3a2a2af73f6 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.694957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-96be238f-a1b0-4d53-8a37-e3a2a2af73f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.767298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be238f-a1b0-4d53-8a37-e3a2a2af73f6 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.772876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78bb82-1e4b-4290-8caa-3be5a4ed1cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.773944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78bb82-1e4b-4290-8caa-3be5a4ed1cfb service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.774237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78bb82-1e4b-4290-8caa-3be5a4ed1cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.774548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78bb82-1e4b-4290-8caa-3be5a4ed1cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.788164 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78bb82-1e4b-4290-8caa-3be5a4ed1cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.790759 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78bb82-1e4b-4290-8caa-3be5a4ed1cfb service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.802122 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1416/2821] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:42.678229 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ffb1d-5cd5-4625-a470-52e35393f7c6 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:42.679084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ffb1d-5cd5-4625-a470-52e35393f7c6 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:42.679567 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ffb1d-5cd5-4625-a470-52e35393f7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:42.718002 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-411ffb1d-5cd5-4625-a470-52e35393f7c6 None None] MFA Rules not processed for user `52bda0133af44d5ca220ba869fbcb3a9`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:42.719601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-411ffb1d-5cd5-4625-a470-52e35393f7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:42.808418 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1406/2822] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:42.822579 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78bb82-1e4b-4290-8caa-3be5a4ed1cfb service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:42.824464 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-5f78bb82-1e4b-4290-8caa-3be5a4ed1cfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:42.891761 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78bb82-1e4b-4290-8caa-3be5a4ed1cfb service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:42.895533 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22d2a1-c0ce-4c99-82dd-f9c2edd24d74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:42.896149 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22d2a1-c0ce-4c99-82dd-f9c2edd24d74 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:42.896423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22d2a1-c0ce-4c99-82dd-f9c2edd24d74 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:42.896693 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22d2a1-c0ce-4c99-82dd-f9c2edd24d74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:42.909274 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22d2a1-c0ce-4c99-82dd-f9c2edd24d74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:42.910934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22d2a1-c0ce-4c99-82dd-f9c2edd24d74 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:42.921454 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1417/2823] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:43.603393 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ffb1d-5cd5-4625-a470-52e35393f7c6 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:43.604850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-411ffb1d-5cd5-4625-a470-52e35393f7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:43.618651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ffb1d-5cd5-4625-a470-52e35393f7c6 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:43.623213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:43.623840 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:43.624118 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:43.624391 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] PATH_INFO: `/v3/users/73e81b4b0a544cd4b9c8b16470b4f41b` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:43.625524 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e81b4b0a544cd4b9c8b16470b4f41b)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:43.627514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:43.676243 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e81b4b0a544cd4b9c8b16470b4f41b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.681394 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:43.685944 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e81b4b0a544cd4b9c8b16470b4f41b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.686949 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e81b4b0a544cd4b9c8b16470b4f41b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.688213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e81b4b0a544cd4b9c8b16470b4f41b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.697459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e81b4b0a544cd4b9c8b16470b4f41b'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.714741 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1407/2824] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:43 2022] DELETE /identity/v3/users/73e81b4b0a544cd4b9c8b16470b4f41b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:43.719503 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e6e75-fca1-4b8a-9885-9b46f31ab725 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:43.719788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e6e75-fca1-4b8a-9885-9b46f31ab725 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:43.720057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e6e75-fca1-4b8a-9885-9b46f31ab725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:43.739519 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-097e6e75-fca1-4b8a-9885-9b46f31ab725 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:43.740795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-097e6e75-fca1-4b8a-9885-9b46f31ab725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:43.812700 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1418/2825] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:43.822601 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:43.823989 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:43.900721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acca5e-8e89-452f-96dd-8a6a6451e7f5 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:43.904463 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3fb2c-b18f-4af7-9752-16f322b9f9db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:43.905342 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3fb2c-b18f-4af7-9752-16f322b9f9db service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:43.905842 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3fb2c-b18f-4af7-9752-16f322b9f9db service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:43.906275 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3fb2c-b18f-4af7-9752-16f322b9f9db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:43.920774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3fb2c-b18f-4af7-9752-16f322b9f9db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:43.922289 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3fb2c-b18f-4af7-9752-16f322b9f9db service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:43.933687 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1408/2826] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:44.267066 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e6e75-fca1-4b8a-9885-9b46f31ab725 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:44.268552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-097e6e75-fca1-4b8a-9885-9b46f31ab725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:44.281439 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e6e75-fca1-4b8a-9885-9b46f31ab725 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:44.290811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae655a5-4db8-43d7-8770-271d4dcf0db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:44.291472 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae655a5-4db8-43d7-8770-271d4dcf0db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.291753 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae655a5-4db8-43d7-8770-271d4dcf0db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.292043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae655a5-4db8-43d7-8770-271d4dcf0db6 admin admin] PATH_INFO: `/v3/projects/57962b6b7b0541589abd69b82a5146e9` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.294860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae655a5-4db8-43d7-8770-271d4dcf0db6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57962b6b7b0541589abd69b82a5146e9)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:44.298143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae655a5-4db8-43d7-8770-271d4dcf0db6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:44.329052 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74dff9-6dcb-425d-a116-85fc006eec5d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.329333 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74dff9-6dcb-425d-a116-85fc006eec5d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.329611 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74dff9-6dcb-425d-a116-85fc006eec5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.350482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-2c74dff9-6dcb-425d-a116-85fc006eec5d None None] MFA Rules not processed for user `52bda0133af44d5ca220ba869fbcb3a9`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:44.351766 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2c74dff9-6dcb-425d-a116-85fc006eec5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:44.353826 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:INSERT=1,keystone:SELECT=97,keystone:DELETE=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:44.367478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-cae655a5-4db8-43d7-8770-271d4dcf0db6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57962b6b7b0541589abd69b82a5146e9', 'request_id': 'req-cae655a5-4db8-43d7-8770-271d4dcf0db6'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:44.377043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-cae655a5-4db8-43d7-8770-271d4dcf0db6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57962b6b7b0541589abd69b82a5146e9', 'request_id': 'req-cae655a5-4db8-43d7-8770-271d4dcf0db6'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:44.386801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-cae655a5-4db8-43d7-8770-271d4dcf0db6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57962b6b7b0541589abd69b82a5146e9', 'request_id': 'req-cae655a5-4db8-43d7-8770-271d4dcf0db6'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:44.399364 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-cae655a5-4db8-43d7-8770-271d4dcf0db6 admin admin] The token cache is being invalidate because project 57962b6b7b0541589abd69b82a5146e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:44.399623 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-cae655a5-4db8-43d7-8770-271d4dcf0db6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:44.401856 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-cae655a5-4db8-43d7-8770-271d4dcf0db6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:44.403152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1419/2827] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:44 2022] DELETE /identity/v3/projects/57962b6b7b0541589abd69b82a5146e9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:44.408112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a17a9-ae34-4a52-989f-080cf9886bf4 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.408423 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a17a9-ae34-4a52-989f-080cf9886bf4 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.408679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a17a9-ae34-4a52-989f-080cf9886bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.428920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1409/2828] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:44.431004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-7b3a17a9-ae34-4a52-989f-080cf9886bf4 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:44.432742 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7b3a17a9-ae34-4a52-989f-080cf9886bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:44.476905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74dff9-6dcb-425d-a116-85fc006eec5d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:44.478431 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2c74dff9-6dcb-425d-a116-85fc006eec5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:44.513112 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1420/2829] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:44.520944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bffd9-275e-4fbe-8345-e70c58f2a426 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.521240 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bffd9-275e-4fbe-8345-e70c58f2a426 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.521516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bffd9-275e-4fbe-8345-e70c58f2a426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.541883 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-4f0bffd9-275e-4fbe-8345-e70c58f2a426 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:44.543091 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4f0bffd9-275e-4fbe-8345-e70c58f2a426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:44.544709 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74dff9-6dcb-425d-a116-85fc006eec5d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:44.548287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fcd40-8cb4-4bcf-9116-579819edbdb3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e27f5b81c704dbebbd6a83b2713e237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:44.548895 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fcd40-8cb4-4bcf-9116-579819edbdb3 service glance] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.549175 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fcd40-8cb4-4bcf-9116-579819edbdb3 service glance] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.549438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47fcd40-8cb4-4bcf-9116-579819edbdb3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.551172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-f47fcd40-8cb4-4bcf-9116-579819edbdb3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:44.551307 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1421/2830] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:44.556231 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bffd9-275e-4fbe-8345-e70c58f2a426 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:44.566406 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bffd9-275e-4fbe-8345-e70c58f2a426 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:44.571092 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8c22d-d3ce-4723-8bbe-bd5b875fae21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:44.571722 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8c22d-d3ce-4723-8bbe-bd5b875fae21 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.572004 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8c22d-d3ce-4723-8bbe-bd5b875fae21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.572279 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8c22d-d3ce-4723-8bbe-bd5b875fae21 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.572932 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8c22d-d3ce-4723-8bbe-bd5b875fae21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:44.573236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8c22d-d3ce-4723-8bbe-bd5b875fae21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:44.574873 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e8c22d-d3ce-4723-8bbe-bd5b875fae21 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:44.584889 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:44.588560 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1422/2831] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:44.593342 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8c22d-d3ce-4723-8bbe-bd5b875fae21 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:44.603834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8c22d-d3ce-4723-8bbe-bd5b875fae21 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:44.608827 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8184a1-f2a2-4281-9e28-455b686e1181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:44.609432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8184a1-f2a2-4281-9e28-455b686e1181 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.609713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8184a1-f2a2-4281-9e28-455b686e1181 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.610016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8184a1-f2a2-4281-9e28-455b686e1181 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.610669 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8184a1-f2a2-4281-9e28-455b686e1181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:44.612577 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8184a1-f2a2-4281-9e28-455b686e1181 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:44.630407 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fcd40-8cb4-4bcf-9116-579819edbdb3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:44.632522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47fcd40-8cb4-4bcf-9116-579819edbdb3 service glance] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:44.642692 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1423/2832] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:44 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:44.646887 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8184a1-f2a2-4281-9e28-455b686e1181 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:44.648233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1410/2833] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:44.714281 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8184a1-f2a2-4281-9e28-455b686e1181 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:44.720552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def39ba-66e2-40cb-92f3-69816ce24aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:44.721271 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def39ba-66e2-40cb-92f3-69816ce24aaf admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.721625 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def39ba-66e2-40cb-92f3-69816ce24aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.722003 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def39ba-66e2-40cb-92f3-69816ce24aaf admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.722706 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def39ba-66e2-40cb-92f3-69816ce24aaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:44.724731 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def39ba-66e2-40cb-92f3-69816ce24aaf admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:44.751126 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1424/2834] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:44 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:44.756262 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fcd40-8cb4-4bcf-9116-579819edbdb3 service glance] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:44.767743 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47fcd40-8cb4-4bcf-9116-579819edbdb3 service glance] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:44.772358 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15043ef8-80ce-456f-a6f7-16214c1383cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:44.772970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15043ef8-80ce-456f-a6f7-16214c1383cd admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.773251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15043ef8-80ce-456f-a6f7-16214c1383cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.773524 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15043ef8-80ce-456f-a6f7-16214c1383cd admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.774206 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15043ef8-80ce-456f-a6f7-16214c1383cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:44.775885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15043ef8-80ce-456f-a6f7-16214c1383cd admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:44.787284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1411/2835] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:44.791939 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def39ba-66e2-40cb-92f3-69816ce24aaf admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:44.801869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def39ba-66e2-40cb-92f3-69816ce24aaf admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:44.807696 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9a603b-fd32-4c9a-a6cd-1d0c485e5a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:44.808394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a603b-fd32-4c9a-a6cd-1d0c485e5a9e admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.808869 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a603b-fd32-4c9a-a6cd-1d0c485e5a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.809166 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9a603b-fd32-4c9a-a6cd-1d0c485e5a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.810199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a603b-fd32-4c9a-a6cd-1d0c485e5a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:44.813777 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9a603b-fd32-4c9a-a6cd-1d0c485e5a9e admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:44.824476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:44.825942 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1425/2836] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:44.828588 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15043ef8-80ce-456f-a6f7-16214c1383cd admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:44.841834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15043ef8-80ce-456f-a6f7-16214c1383cd admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:44.846686 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911551a-466b-4210-9988-a5e73f30b5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:44.847344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911551a-466b-4210-9988-a5e73f30b5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.847744 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911551a-466b-4210-9988-a5e73f30b5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.848206 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911551a-466b-4210-9988-a5e73f30b5b7 admin admin] PATH_INFO: `/v3/projects/838f4b9739be4a289ef662deb82f9939/users/1b2474c9803f4c3da14b787704ca6744/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.867307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911551a-466b-4210-9988-a5e73f30b5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=838f4b9739be4a289ef662deb82f9939, user_id=1b2474c9803f4c3da14b787704ca6744, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:44.870775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911551a-466b-4210-9988-a5e73f30b5b7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:44.889251 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1412/2837] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:44 2022] PUT /identity/v3/projects/838f4b9739be4a289ef662deb82f9939/users/1b2474c9803f4c3da14b787704ca6744/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:44.900040 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a4e95-8980-4a48-8aed-135a583a4b02 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.900335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a4e95-8980-4a48-8aed-135a583a4b02 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.900612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a4e95-8980-4a48-8aed-135a583a4b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.945459 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-3c2a4e95-8980-4a48-8aed-135a583a4b02 None None] MFA Rules not processed for user `1b2474c9803f4c3da14b787704ca6744`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:44.946757 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-3c2a4e95-8980-4a48-8aed-135a583a4b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:45.028337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1426/2838] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:45.032837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911551a-466b-4210-9988-a5e73f30b5b7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.101511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911551a-466b-4210-9988-a5e73f30b5b7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.106382 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc72fa-817c-4062-9744-1411bd27084d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.106850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc72fa-817c-4062-9744-1411bd27084d admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.107227 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc72fa-817c-4062-9744-1411bd27084d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.107501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc72fa-817c-4062-9744-1411bd27084d admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.108235 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc72fa-817c-4062-9744-1411bd27084d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:45.110356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc72fa-817c-4062-9744-1411bd27084d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:45.150701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1413/2839] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:45 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:45.151639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a4e95-8980-4a48-8aed-135a583a4b02 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.162578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a4e95-8980-4a48-8aed-135a583a4b02 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.242470 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8380800-71e3-441c-b96e-dbb0fdaf8abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.243143 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8380800-71e3-441c-b96e-dbb0fdaf8abd admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.243421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8380800-71e3-441c-b96e-dbb0fdaf8abd admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.243690 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8380800-71e3-441c-b96e-dbb0fdaf8abd admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.244330 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8380800-71e3-441c-b96e-dbb0fdaf8abd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:45.248517 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8380800-71e3-441c-b96e-dbb0fdaf8abd admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:45.263103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc72fa-817c-4062-9744-1411bd27084d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.266348 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-20bc72fa-817c-4062-9744-1411bd27084d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:45.275095 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1427/2840] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:45 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:45.282458 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8380800-71e3-441c-b96e-dbb0fdaf8abd admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.295843 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8380800-71e3-441c-b96e-dbb0fdaf8abd admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.300765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d2190-746a-4b40-8771-26f9d74bccf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.301366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2190-746a-4b40-8771-26f9d74bccf9 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.301638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2190-746a-4b40-8771-26f9d74bccf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.301955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2190-746a-4b40-8771-26f9d74bccf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.302804 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d2190-746a-4b40-8771-26f9d74bccf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:45.304213 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d2190-746a-4b40-8771-26f9d74bccf9 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:45.314727 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1428/2841] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:45.319527 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d2190-746a-4b40-8771-26f9d74bccf9 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.330552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d2190-746a-4b40-8771-26f9d74bccf9 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.335349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c16346-a32c-437f-8aa9-27af381ab53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.335946 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c16346-a32c-437f-8aa9-27af381ab53f admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.336223 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c16346-a32c-437f-8aa9-27af381ab53f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.336488 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c16346-a32c-437f-8aa9-27af381ab53f admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.337791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c16346-a32c-437f-8aa9-27af381ab53f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:45.339590 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c16346-a32c-437f-8aa9-27af381ab53f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:45.339727 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc72fa-817c-4062-9744-1411bd27084d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.344494 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2412ea-069c-4bb2-bbde-2e46a58fcb81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.345215 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2412ea-069c-4bb2-bbde-2e46a58fcb81 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.345562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2412ea-069c-4bb2-bbde-2e46a58fcb81 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.345983 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2412ea-069c-4bb2-bbde-2e46a58fcb81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.347851 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-9c2412ea-069c-4bb2-bbde-2e46a58fcb81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:45.357173 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1429/2842] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:45.358374 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c16346-a32c-437f-8aa9-27af381ab53f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.370573 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c16346-a32c-437f-8aa9-27af381ab53f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.376920 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:45.378297 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b361a32-aa5a-448c-a0dc-313cccd82b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.380476 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b361a32-aa5a-448c-a0dc-313cccd82b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.380987 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b361a32-aa5a-448c-a0dc-313cccd82b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.381281 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b361a32-aa5a-448c-a0dc-313cccd82b75 admin admin] PATH_INFO: `/v3/projects/0e85c1bf28574ac6a73a047aa037e358/users/132ebabdc2df4d298ac80c8f2b45a536/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.401315 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b361a32-aa5a-448c-a0dc-313cccd82b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e85c1bf28574ac6a73a047aa037e358, user_id=132ebabdc2df4d298ac80c8f2b45a536, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:45.404106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b361a32-aa5a-448c-a0dc-313cccd82b75 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:45.422316 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1430/2843] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:45 2022] PUT /identity/v3/projects/0e85c1bf28574ac6a73a047aa037e358/users/132ebabdc2df4d298ac80c8f2b45a536/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:45.433874 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb2b92-84d1-49d6-9411-27bde16c4571 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.434159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb2b92-84d1-49d6-9411-27bde16c4571 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.434431 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb2b92-84d1-49d6-9411-27bde16c4571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.445897 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2412ea-069c-4bb2-bbde-2e46a58fcb81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:45.447899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2412ea-069c-4bb2-bbde-2e46a58fcb81 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:45.460280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1414/2844] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:45.473769 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-87fb2b92-84d1-49d6-9411-27bde16c4571 None None] MFA Rules not processed for user `132ebabdc2df4d298ac80c8f2b45a536`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:45.475088 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-87fb2b92-84d1-49d6-9411-27bde16c4571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:45.504819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:45.561798 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1431/2845] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:45.566929 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2412ea-069c-4bb2-bbde-2e46a58fcb81 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.578406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2412ea-069c-4bb2-bbde-2e46a58fcb81 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.641448 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f05c78-4115-4929-a4f5-4cecdba9277a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.642123 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f05c78-4115-4929-a4f5-4cecdba9277a admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.642406 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f05c78-4115-4929-a4f5-4cecdba9277a admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.642679 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f05c78-4115-4929-a4f5-4cecdba9277a admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.643322 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f05c78-4115-4929-a4f5-4cecdba9277a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:45.645329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f05c78-4115-4929-a4f5-4cecdba9277a admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:45.676715 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1415/2846] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:45 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:45.681475 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb2b92-84d1-49d6-9411-27bde16c4571 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.692879 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb2b92-84d1-49d6-9411-27bde16c4571 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.753610 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00ac7f-3a0e-4dee-9eea-d0c20b12263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.754260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00ac7f-3a0e-4dee-9eea-d0c20b12263f admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.754546 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00ac7f-3a0e-4dee-9eea-d0c20b12263f admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.754819 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00ac7f-3a0e-4dee-9eea-d0c20b12263f admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.755501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00ac7f-3a0e-4dee-9eea-d0c20b12263f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:45.757625 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00ac7f-3a0e-4dee-9eea-d0c20b12263f admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:45.782599 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1432/2847] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:45 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:45.786734 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f05c78-4115-4929-a4f5-4cecdba9277a admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.800853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f05c78-4115-4929-a4f5-4cecdba9277a admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.806094 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca528b6a-a50c-471d-9c6c-2d1ce42f8648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.806706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca528b6a-a50c-471d-9c6c-2d1ce42f8648 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.806988 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca528b6a-a50c-471d-9c6c-2d1ce42f8648 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.807263 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca528b6a-a50c-471d-9c6c-2d1ce42f8648 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.807909 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca528b6a-a50c-471d-9c6c-2d1ce42f8648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:45.809504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca528b6a-a50c-471d-9c6c-2d1ce42f8648 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:45.821131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1416/2848] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:45.826032 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00ac7f-3a0e-4dee-9eea-d0c20b12263f admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.837437 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00ac7f-3a0e-4dee-9eea-d0c20b12263f admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.843034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb2347-5881-4f8d-a3fb-7bd10114f9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.843797 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb2347-5881-4f8d-a3fb-7bd10114f9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.844081 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb2347-5881-4f8d-a3fb-7bd10114f9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.844450 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb2347-5881-4f8d-a3fb-7bd10114f9e6 admin admin] PATH_INFO: `/v3/projects/94466f5d731842c0bf01a0b5ce508130/users/9c827351748b419a949b01643fc4a0bd/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.861909 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb2347-5881-4f8d-a3fb-7bd10114f9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94466f5d731842c0bf01a0b5ce508130, user_id=9c827351748b419a949b01643fc4a0bd, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:45.864785 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb2347-5881-4f8d-a3fb-7bd10114f9e6 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:45.881765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1433/2849] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:45 2022] PUT /identity/v3/projects/94466f5d731842c0bf01a0b5ce508130/users/9c827351748b419a949b01643fc4a0bd/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:45.893428 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca528b6a-a50c-471d-9c6c-2d1ce42f8648 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.927834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca528b6a-a50c-471d-9c6c-2d1ce42f8648 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.999171 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83702fdc-1312-4788-92d6-40e9e97c16de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.999985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83702fdc-1312-4788-92d6-40e9e97c16de admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:46.000270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83702fdc-1312-4788-92d6-40e9e97c16de admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:46.000542 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83702fdc-1312-4788-92d6-40e9e97c16de admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:46.001189 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83702fdc-1312-4788-92d6-40e9e97c16de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:46.002855 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83702fdc-1312-4788-92d6-40e9e97c16de admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:46.016069 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1417/2850] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:46.020377 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb2347-5881-4f8d-a3fb-7bd10114f9e6 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:46.031944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb2347-5881-4f8d-a3fb-7bd10114f9e6 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:46.038295 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66467480-7909-4788-89d6-745f744bd851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:46.043304 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66467480-7909-4788-89d6-745f744bd851 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:46.043582 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66467480-7909-4788-89d6-745f744bd851 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:46.043860 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66467480-7909-4788-89d6-745f744bd851 admin admin] PATH_INFO: `/v3/projects/94466f5d731842c0bf01a0b5ce508130/users/9c827351748b419a949b01643fc4a0bd/roles/398624259b8d4e2ea5beebc4f2ecbf57` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:46.046026 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66467480-7909-4788-89d6-745f744bd851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94466f5d731842c0bf01a0b5ce508130, user_id=9c827351748b419a949b01643fc4a0bd, role_id=398624259b8d4e2ea5beebc4f2ecbf57)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:46.048654 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66467480-7909-4788-89d6-745f744bd851 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:46.064606 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1434/2851] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:46 2022] PUT /identity/v3/projects/94466f5d731842c0bf01a0b5ce508130/users/9c827351748b419a949b01643fc4a0bd/roles/398624259b8d4e2ea5beebc4f2ecbf57 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:46.075335 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08a8e4-9137-4e32-add2-6b72ad12935d None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:46.075704 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08a8e4-9137-4e32-add2-6b72ad12935d None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:46.076064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08a8e4-9137-4e32-add2-6b72ad12935d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:46.107640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-4e08a8e4-9137-4e32-add2-6b72ad12935d None None] MFA Rules not processed for user `9c827351748b419a949b01643fc4a0bd`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:46.109509 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-4e08a8e4-9137-4e32-add2-6b72ad12935d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:46.119197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:46.180233 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1418/2852] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:46.192113 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66467480-7909-4788-89d6-745f744bd851 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:46.193867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-66467480-7909-4788-89d6-745f744bd851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:46.254571 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66467480-7909-4788-89d6-745f744bd851 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:46.258257 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cf2d3-569c-4cb6-acfa-6a92b0197e29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:46.258900 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cf2d3-569c-4cb6-acfa-6a92b0197e29 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:46.259176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cf2d3-569c-4cb6-acfa-6a92b0197e29 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:46.259445 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701cf2d3-569c-4cb6-acfa-6a92b0197e29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:46.288738 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:46.319229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cf2d3-569c-4cb6-acfa-6a92b0197e29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:46.320652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701cf2d3-569c-4cb6-acfa-6a92b0197e29 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:46.329419 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1435/2853] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:47.968343 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08a8e4-9137-4e32-add2-6b72ad12935d None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:47.979287 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08a8e4-9137-4e32-add2-6b72ad12935d None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:47.983145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0118dd-1617-48c6-8318-645a1c98691f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:47.983764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0118dd-1617-48c6-8318-645a1c98691f service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:47.984037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0118dd-1617-48c6-8318-645a1c98691f service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:47.984307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0118dd-1617-48c6-8318-645a1c98691f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:48.058590 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0118dd-1617-48c6-8318-645a1c98691f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:48.060436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0118dd-1617-48c6-8318-645a1c98691f service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:48.076107 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1419/2854] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:48.667916 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cf2d3-569c-4cb6-acfa-6a92b0197e29 service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:48.678041 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701cf2d3-569c-4cb6-acfa-6a92b0197e29 service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:48.682333 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac58a1-a9f2-40d0-9dfc-4f8090b02fe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:48.682963 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac58a1-a9f2-40d0-9dfc-4f8090b02fe1 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:48.683242 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac58a1-a9f2-40d0-9dfc-4f8090b02fe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:48.683555 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac58a1-a9f2-40d0-9dfc-4f8090b02fe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:48.693398 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac58a1-a9f2-40d0-9dfc-4f8090b02fe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:48.694906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac58a1-a9f2-40d0-9dfc-4f8090b02fe1 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:48.706624 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1436/2855] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:52.659701 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0118dd-1617-48c6-8318-645a1c98691f service cinder] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:52.739393 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0118dd-1617-48c6-8318-645a1c98691f service cinder] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:52.744670 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:52.745531 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:52.745925 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:52.746356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] PATH_INFO: `/v3/users/1b2474c9803f4c3da14b787704ca6744` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:52.747646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b2474c9803f4c3da14b787704ca6744)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:52.749870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:52.815349 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:52.817418 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b2474c9803f4c3da14b787704ca6744'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:52.826459 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b2474c9803f4c3da14b787704ca6744'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:52.827424 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b2474c9803f4c3da14b787704ca6744'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:52.828058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b2474c9803f4c3da14b787704ca6744'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:52.836524 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b2474c9803f4c3da14b787704ca6744'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:52.855860 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1420/2856] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:52 2022] DELETE /identity/v3/users/1b2474c9803f4c3da14b787704ca6744 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:52.860495 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d76d1-3826-4963-b452-cf9911d42bdb None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:52.860962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d76d1-3826-4963-b452-cf9911d42bdb None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:52.861076 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909d76d1-3826-4963-b452-cf9911d42bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:52.881934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-909d76d1-3826-4963-b452-cf9911d42bdb None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:52.883153 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-909d76d1-3826-4963-b452-cf9911d42bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:52.948019 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1437/2857] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:52.958785 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:52.960461 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:53.026756 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ee323-eb62-4013-a249-8b7d80886ac2 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.030368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774b089-38ed-473d-aa40-3ac0b3a06578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.030970 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774b089-38ed-473d-aa40-3ac0b3a06578 service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.031249 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774b089-38ed-473d-aa40-3ac0b3a06578 service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.031515 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774b089-38ed-473d-aa40-3ac0b3a06578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.041188 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774b089-38ed-473d-aa40-3ac0b3a06578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.042649 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774b089-38ed-473d-aa40-3ac0b3a06578 service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.053011 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1421/2858] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:53.359997 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d76d1-3826-4963-b452-cf9911d42bdb None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.361541 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-909d76d1-3826-4963-b452-cf9911d42bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:53.387132 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909d76d1-3826-4963-b452-cf9911d42bdb None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.394236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.395102 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.395381 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.395750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] PATH_INFO: `/v3/projects/838f4b9739be4a289ef662deb82f9939` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.403867 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=838f4b9739be4a289ef662deb82f9939)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.405959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.452947 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '838f4b9739be4a289ef662deb82f9939', 'request_id': 'req-17070838-fd86-447e-9482-b8ebb94da275'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:53.459349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '838f4b9739be4a289ef662deb82f9939', 'request_id': 'req-17070838-fd86-447e-9482-b8ebb94da275'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:53.467580 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '838f4b9739be4a289ef662deb82f9939', 'request_id': 'req-17070838-fd86-447e-9482-b8ebb94da275'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:53.478570 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] The token cache is being invalidate because project 838f4b9739be4a289ef662deb82f9939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:53.478852 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:53.479457 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:53.480953 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1438/2859] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:53 2022] DELETE /identity/v3/projects/838f4b9739be4a289ef662deb82f9939 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:53.484936 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774b089-38ed-473d-aa40-3ac0b3a06578 service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.494317 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774b089-38ed-473d-aa40-3ac0b3a06578 service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.550308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.550937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.551218 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.551492 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] PATH_INFO: `/v3/users/132ebabdc2df4d298ac80c8f2b45a536` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.552625 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=132ebabdc2df4d298ac80c8f2b45a536)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.554582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.601802 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '132ebabdc2df4d298ac80c8f2b45a536'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:53.609120 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '132ebabdc2df4d298ac80c8f2b45a536'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:53.609810 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '132ebabdc2df4d298ac80c8f2b45a536'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:53.610441 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '132ebabdc2df4d298ac80c8f2b45a536'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:53.617224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '132ebabdc2df4d298ac80c8f2b45a536'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:53.621506 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:53.636483 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1422/2860] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:53 2022] DELETE /identity/v3/users/132ebabdc2df4d298ac80c8f2b45a536 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:53.972315 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.973818 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:53.986493 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17070838-fd86-447e-9482-b8ebb94da275 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:54.036422 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:54.059959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:54.060601 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:54.060876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:54.061144 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] PATH_INFO: `/v3/projects/0e85c1bf28574ac6a73a047aa037e358` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:54.062284 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e85c1bf28574ac6a73a047aa037e358)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:54.064233 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:54.110108 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e85c1bf28574ac6a73a047aa037e358', 'request_id': 'req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.116309 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e85c1bf28574ac6a73a047aa037e358', 'request_id': 'req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.123965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e85c1bf28574ac6a73a047aa037e358', 'request_id': 'req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.134751 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] The token cache is being invalidate because project 0e85c1bf28574ac6a73a047aa037e358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:54.135034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.135652 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.137126 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1439/2861] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:53 2022] DELETE /identity/v3/projects/0e85c1bf28574ac6a73a047aa037e358 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:54.141224 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:54.142640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:54.155164 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a8b170-589b-4fc3-ba98-988360fbaa68 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:54.210329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a4d87-9528-4a5a-b366-18a8127b6a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:54.210955 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4d87-9528-4a5a-b366-18a8127b6a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:54.211238 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4d87-9528-4a5a-b366-18a8127b6a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:54.211535 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a4d87-9528-4a5a-b366-18a8127b6a3e admin admin] PATH_INFO: `/v3/users/9c827351748b419a949b01643fc4a0bd` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:54.212637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a4d87-9528-4a5a-b366-18a8127b6a3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c827351748b419a949b01643fc4a0bd)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:54.214587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a4d87-9528-4a5a-b366-18a8127b6a3e admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:54.263543 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-272a4d87-9528-4a5a-b366-18a8127b6a3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c827351748b419a949b01643fc4a0bd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.272145 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-272a4d87-9528-4a5a-b366-18a8127b6a3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c827351748b419a949b01643fc4a0bd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.272947 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-272a4d87-9528-4a5a-b366-18a8127b6a3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c827351748b419a949b01643fc4a0bd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.273674 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-272a4d87-9528-4a5a-b366-18a8127b6a3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c827351748b419a949b01643fc4a0bd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.280732 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-272a4d87-9528-4a5a-b366-18a8127b6a3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c827351748b419a949b01643fc4a0bd'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.297936 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1423/2862] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:54 2022] DELETE /identity/v3/users/9c827351748b419a949b01643fc4a0bd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:54.572043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:54.573461 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:54.587371 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e5d63d-9cde-4c4e-8c21-11d0fbe72d22 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:54.646456 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb85d8a-4826-47b6-bd0a-551b2c919d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:54.647007 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb85d8a-4826-47b6-bd0a-551b2c919d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:54.647284 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb85d8a-4826-47b6-bd0a-551b2c919d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:54.647557 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb85d8a-4826-47b6-bd0a-551b2c919d30 admin admin] PATH_INFO: `/v3/projects/94466f5d731842c0bf01a0b5ce508130` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:54.648679 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb85d8a-4826-47b6-bd0a-551b2c919d30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94466f5d731842c0bf01a0b5ce508130)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:54.651179 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb85d8a-4826-47b6-bd0a-551b2c919d30 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:54.697090 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4eb85d8a-4826-47b6-bd0a-551b2c919d30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94466f5d731842c0bf01a0b5ce508130', 'request_id': 'req-4eb85d8a-4826-47b6-bd0a-551b2c919d30'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.703401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4eb85d8a-4826-47b6-bd0a-551b2c919d30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94466f5d731842c0bf01a0b5ce508130', 'request_id': 'req-4eb85d8a-4826-47b6-bd0a-551b2c919d30'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.710469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4eb85d8a-4826-47b6-bd0a-551b2c919d30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94466f5d731842c0bf01a0b5ce508130', 'request_id': 'req-4eb85d8a-4826-47b6-bd0a-551b2c919d30'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.718054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:54.721854 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4eb85d8a-4826-47b6-bd0a-551b2c919d30 admin admin] The token cache is being invalidate because project 94466f5d731842c0bf01a0b5ce508130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:54.722320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4eb85d8a-4826-47b6-bd0a-551b2c919d30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.723260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-4eb85d8a-4826-47b6-bd0a-551b2c919d30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:54.724718 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1440/2863] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:54 2022] DELETE /identity/v3/projects/94466f5d731842c0bf01a0b5ce508130 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:54.730792 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57a70f-4633-42ee-9fe3-be08d4afe719 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:54.731088 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57a70f-4633-42ee-9fe3-be08d4afe719 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:54.731368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57a70f-4633-42ee-9fe3-be08d4afe719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:54.752122 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-6b57a70f-4633-42ee-9fe3-be08d4afe719 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:54.753458 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-6b57a70f-4633-42ee-9fe3-be08d4afe719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:54.816178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1424/2864] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:54.881548 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e36834-0e9f-4131-ba38-618ddca7fbfb None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:54.881895 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e36834-0e9f-4131-ba38-618ddca7fbfb None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:54.882173 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e36834-0e9f-4131-ba38-618ddca7fbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:54.902481 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-74e36834-0e9f-4131-ba38-618ddca7fbfb None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:54.903611 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-74e36834-0e9f-4131-ba38-618ddca7fbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:54.912087 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1441/2865] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:54.916268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57a70f-4633-42ee-9fe3-be08d4afe719 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:54.922228 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:54.926031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57a70f-4633-42ee-9fe3-be08d4afe719 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:54.932436 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba546a-436b-46e2-9d18-b8b9153cd1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:54.933209 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba546a-436b-46e2-9d18-b8b9153cd1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:54.933665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba546a-436b-46e2-9d18-b8b9153cd1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:54.934031 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba546a-436b-46e2-9d18-b8b9153cd1a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:54.934899 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba546a-436b-46e2-9d18-b8b9153cd1a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:54.935214 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba546a-436b-46e2-9d18-b8b9153cd1a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:54.936878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba546a-436b-46e2-9d18-b8b9153cd1a9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:54.948960 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1425/2866] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:54.953229 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e36834-0e9f-4131-ba38-618ddca7fbfb None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:54.962353 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e36834-0e9f-4131-ba38-618ddca7fbfb None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:54.967385 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e2989-b173-4cac-8a2c-904d8da54456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:54.967978 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e2989-b173-4cac-8a2c-904d8da54456 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:54.968260 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e2989-b173-4cac-8a2c-904d8da54456 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:54.968530 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e2989-b173-4cac-8a2c-904d8da54456 admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:54.969151 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e2989-b173-4cac-8a2c-904d8da54456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:54.971062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e2989-b173-4cac-8a2c-904d8da54456 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:54.997830 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1442/2867] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:54 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:55.002051 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba546a-436b-46e2-9d18-b8b9153cd1a9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:55.012378 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba546a-436b-46e2-9d18-b8b9153cd1a9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:55.071178 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee4af7-d40e-43ee-aaf5-edc97a9b19b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:55.071801 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee4af7-d40e-43ee-aaf5-edc97a9b19b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:55.072074 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee4af7-d40e-43ee-aaf5-edc97a9b19b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:55.072340 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee4af7-d40e-43ee-aaf5-edc97a9b19b7 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:55.072987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee4af7-d40e-43ee-aaf5-edc97a9b19b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:55.074910 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee4af7-d40e-43ee-aaf5-edc97a9b19b7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:55.096926 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1426/2868] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:55 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:55.101165 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e2989-b173-4cac-8a2c-904d8da54456 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:55.111911 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e2989-b173-4cac-8a2c-904d8da54456 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:55.116618 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cab2f4-79b3-4d7f-a74c-f7306af1c20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:55.117255 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab2f4-79b3-4d7f-a74c-f7306af1c20c admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:55.117542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab2f4-79b3-4d7f-a74c-f7306af1c20c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:55.118394 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cab2f4-79b3-4d7f-a74c-f7306af1c20c admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:55.119061 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cab2f4-79b3-4d7f-a74c-f7306af1c20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:55.120645 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cab2f4-79b3-4d7f-a74c-f7306af1c20c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:55.133326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1443/2869] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:55.135305 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee4af7-d40e-43ee-aaf5-edc97a9b19b7 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:55.145621 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee4af7-d40e-43ee-aaf5-edc97a9b19b7 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:55.150315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd3737-90ca-42f6-9ba7-f2e51df661f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:55.150919 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd3737-90ca-42f6-9ba7-f2e51df661f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:55.151198 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd3737-90ca-42f6-9ba7-f2e51df661f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:55.151471 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd3737-90ca-42f6-9ba7-f2e51df661f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:55.152116 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd3737-90ca-42f6-9ba7-f2e51df661f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:55.153775 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd3737-90ca-42f6-9ba7-f2e51df661f7 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:55.165559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1427/2870] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:55.169850 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cab2f4-79b3-4d7f-a74c-f7306af1c20c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:55.179691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cab2f4-79b3-4d7f-a74c-f7306af1c20c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:55.184815 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83b720-6e23-40d6-8a27-e4b6073cb80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:55.185470 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83b720-6e23-40d6-8a27-e4b6073cb80c admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:55.185765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83b720-6e23-40d6-8a27-e4b6073cb80c admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:55.186046 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83b720-6e23-40d6-8a27-e4b6073cb80c admin admin] PATH_INFO: `/v3/projects/5e00b70b79844467a746eab7407353e3/users/6f5a9021e2ad4561a2cdf89a1f98204c/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:55.201160 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83b720-6e23-40d6-8a27-e4b6073cb80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e00b70b79844467a746eab7407353e3, user_id=6f5a9021e2ad4561a2cdf89a1f98204c, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:55.203812 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83b720-6e23-40d6-8a27-e4b6073cb80c admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:55.218480 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1444/2871] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:55 2022] PUT /identity/v3/projects/5e00b70b79844467a746eab7407353e3/users/6f5a9021e2ad4561a2cdf89a1f98204c/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:55.228644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198fe23-b627-409b-9852-8c8c9b45993a None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:55.228930 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198fe23-b627-409b-9852-8c8c9b45993a None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:55.229207 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198fe23-b627-409b-9852-8c8c9b45993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:55.256852 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-5198fe23-b627-409b-9852-8c8c9b45993a None None] MFA Rules not processed for user `6f5a9021e2ad4561a2cdf89a1f98204c`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:55.258087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-5198fe23-b627-409b-9852-8c8c9b45993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:55.322728 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1428/2872] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:55.334194 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83b720-6e23-40d6-8a27-e4b6073cb80c admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:55.336053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-9b83b720-6e23-40d6-8a27-e4b6073cb80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:55.397529 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83b720-6e23-40d6-8a27-e4b6073cb80c admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:55.401750 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b88f2b-3969-4332-baf5-6e459e9db01c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:55.402364 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b88f2b-3969-4332-baf5-6e459e9db01c service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:55.402638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b88f2b-3969-4332-baf5-6e459e9db01c service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:55.402906 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b88f2b-3969-4332-baf5-6e459e9db01c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:55.415105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b88f2b-3969-4332-baf5-6e459e9db01c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:55.415105 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b88f2b-3969-4332-baf5-6e459e9db01c service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:55.422779 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1445/2873] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:59.549485 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5b7cb-322a-41ab-b1ff-c752c3c439ea None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:59.550040 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5b7cb-322a-41ab-b1ff-c752c3c439ea None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:59.550129 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5b7cb-322a-41ab-b1ff-c752c3c439ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:59.570847 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-0db5b7cb-322a-41ab-b1ff-c752c3c439ea None None] MFA Rules not processed for user `52bda0133af44d5ca220ba869fbcb3a9`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:59.572130 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0db5b7cb-322a-41ab-b1ff-c752c3c439ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:59.639520 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1429/2874] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:02:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:59.684940 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b88f2b-3969-4332-baf5-6e459e9db01c service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:59.686955 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f9b88f2b-3969-4332-baf5-6e459e9db01c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:59.775131 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b88f2b-3969-4332-baf5-6e459e9db01c service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:59.780401 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a56798-9ca7-4b19-9fc8-7de84741dcf5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:59.781010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a56798-9ca7-4b19-9fc8-7de84741dcf5 service placement] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:59.781292 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a56798-9ca7-4b19-9fc8-7de84741dcf5 service placement] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:59.781558 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a56798-9ca7-4b19-9fc8-7de84741dcf5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:59.783331 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-25a56798-9ca7-4b19-9fc8-7de84741dcf5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:59.795501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a56798-9ca7-4b19-9fc8-7de84741dcf5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:59.797012 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a56798-9ca7-4b19-9fc8-7de84741dcf5 service placement] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:59.807842 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1446/2875] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:01.909787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5b7cb-322a-41ab-b1ff-c752c3c439ea None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:01.912208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-0db5b7cb-322a-41ab-b1ff-c752c3c439ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:01.920825 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:01.978140 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5b7cb-322a-41ab-b1ff-c752c3c439ea None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:01.982838 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d4bb6-c050-4505-8b74-3d2d5ee38b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:01.983566 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d4bb6-c050-4505-8b74-3d2d5ee38b9a service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:01.984032 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d4bb6-c050-4505-8b74-3d2d5ee38b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:01.984347 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d4bb6-c050-4505-8b74-3d2d5ee38b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:01.986599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-8f8d4bb6-c050-4505-8b74-3d2d5ee38b9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:02.000369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d4bb6-c050-4505-8b74-3d2d5ee38b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:02.001863 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d4bb6-c050-4505-8b74-3d2d5ee38b9a service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:02.011279 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1430/2876] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:09.803584 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=85,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:12.006955 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=24 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:16.447882 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a56798-9ca7-4b19-9fc8-7de84741dcf5 service placement] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:16.457074 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a56798-9ca7-4b19-9fc8-7de84741dcf5 service placement] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:16.460650 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fc2bf-5373-4e0c-b4ab-c5d8777377b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:16.461241 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fc2bf-5373-4e0c-b4ab-c5d8777377b0 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:16.461511 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fc2bf-5373-4e0c-b4ab-c5d8777377b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:16.462169 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fc2bf-5373-4e0c-b4ab-c5d8777377b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:16.463578 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-aa6fc2bf-5373-4e0c-b4ab-c5d8777377b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:16.550638 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6fc2bf-5373-4e0c-b4ab-c5d8777377b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:16.552397 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6fc2bf-5373-4e0c-b4ab-c5d8777377b0 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:16.563612 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1447/2877] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:24.492417 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96296b3-2b6f-4b24-80b4-048ef9ae236e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:24.493366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96296b3-2b6f-4b24-80b4-048ef9ae236e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:24.493651 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96296b3-2b6f-4b24-80b4-048ef9ae236e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:24.513008 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e96296b3-2b6f-4b24-80b4-048ef9ae236e None None] MFA Rules not processed for user `76a9dc3ddb144e599a2856fce8dc339d`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:24.514239 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e96296b3-2b6f-4b24-80b4-048ef9ae236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:24.523398 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1431/2878] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:03:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:26.053368 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fc2bf-5373-4e0c-b4ab-c5d8777377b0 service cinder] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:26.062848 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6fc2bf-5373-4e0c-b4ab-c5d8777377b0 service cinder] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:26.067332 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8aa01-35d9-4ef4-847c-daa6474659e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:26.068034 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8aa01-35d9-4ef4-847c-daa6474659e9 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:26.068490 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8aa01-35d9-4ef4-847c-daa6474659e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:26.068758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8aa01-35d9-4ef4-847c-daa6474659e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:26.071031 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-09b8aa01-35d9-4ef4-847c-daa6474659e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:26.136832 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8aa01-35d9-4ef4-847c-daa6474659e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:26.138382 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8aa01-35d9-4ef4-847c-daa6474659e9 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:26.148038 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1448/2879] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:28.118731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96296b3-2b6f-4b24-80b4-048ef9ae236e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:28.120141 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e96296b3-2b6f-4b24-80b4-048ef9ae236e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:28.213946 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96296b3-2b6f-4b24-80b4-048ef9ae236e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:28.220213 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32167b-b49f-42c6-8de6-247a3318b6e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:28.220987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32167b-b49f-42c6-8de6-247a3318b6e3 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:28.221326 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32167b-b49f-42c6-8de6-247a3318b6e3 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:28.221599 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32167b-b49f-42c6-8de6-247a3318b6e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:28.223806 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2a32167b-b49f-42c6-8de6-247a3318b6e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:28.237546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32167b-b49f-42c6-8de6-247a3318b6e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:28.239339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a32167b-b49f-42c6-8de6-247a3318b6e3 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:28.251182 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1432/2880] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:03:33.991329 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59663e-0867-45f7-b54e-cc2ce613200b None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:33.992390 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59663e-0867-45f7-b54e-cc2ce613200b None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:33.992858 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59663e-0867-45f7-b54e-cc2ce613200b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:34.046236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-6b59663e-0867-45f7-b54e-cc2ce613200b None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:34.047451 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-6b59663e-0867-45f7-b54e-cc2ce613200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:34.114339 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1449/2881] 10.222.0.55 () {58 vars in 1009 bytes} [Thu Sep 8 14:03:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:34.161326 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32167b-b49f-42c6-8de6-247a3318b6e3 service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:34.162799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-2a32167b-b49f-42c6-8de6-247a3318b6e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:34.253626 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32167b-b49f-42c6-8de6-247a3318b6e3 service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:34.258676 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546c421-db7a-40fb-a2f9-db4372c87773 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:34.259280 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546c421-db7a-40fb-a2f9-db4372c87773 service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:34.259562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546c421-db7a-40fb-a2f9-db4372c87773 service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:34.259836 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546c421-db7a-40fb-a2f9-db4372c87773 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:34.267815 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546c421-db7a-40fb-a2f9-db4372c87773 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:34.269876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546c421-db7a-40fb-a2f9-db4372c87773 service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:34.278512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1433/2882] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 14:03:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 14:03:40.547123 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59663e-0867-45f7-b54e-cc2ce613200b None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:40.552071 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=99 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:40.623432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59663e-0867-45f7-b54e-cc2ce613200b None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:40.628564 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:40.629428 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:40.629709 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:40.630019 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] PATH_INFO: `/v3/users/6f5a9021e2ad4561a2cdf89a1f98204c` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:40.631175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f5a9021e2ad4561a2cdf89a1f98204c)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:40.633675 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:40.688275 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f5a9021e2ad4561a2cdf89a1f98204c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:40.698478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f5a9021e2ad4561a2cdf89a1f98204c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:40.699314 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f5a9021e2ad4561a2cdf89a1f98204c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:40.700062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f5a9021e2ad4561a2cdf89a1f98204c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:40.707862 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f5a9021e2ad4561a2cdf89a1f98204c'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:40.729016 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1450/2883] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:40 2022] DELETE /identity/v3/users/6f5a9021e2ad4561a2cdf89a1f98204c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:40.733574 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5e37f-12b7-4df0-9e14-7edb227c63dc None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:40.734111 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5e37f-12b7-4df0-9e14-7edb227c63dc None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:40.734514 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a5e37f-12b7-4df0-9e14-7edb227c63dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:40.756752 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-82a5e37f-12b7-4df0-9e14-7edb227c63dc None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:40.758794 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-82a5e37f-12b7-4df0-9e14-7edb227c63dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:40.809568 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=100 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:40.827625 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1434/2884] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:40.838130 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:40.839701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:40.908346 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522fe5c6-6962-490b-a9cc-47b006b98b00 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:40.913299 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f390a05-f05b-4582-ad62-66f403dc7086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:40.914658 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f390a05-f05b-4582-ad62-66f403dc7086 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:40.914959 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f390a05-f05b-4582-ad62-66f403dc7086 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:40.915230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f390a05-f05b-4582-ad62-66f403dc7086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:40.925901 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f390a05-f05b-4582-ad62-66f403dc7086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:40.927414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f390a05-f05b-4582-ad62-66f403dc7086 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:40.943991 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1451/2885] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:41.289640 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5e37f-12b7-4df0-9e14-7edb227c63dc None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:41.291892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-82a5e37f-12b7-4df0-9e14-7edb227c63dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:41.307038 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a5e37f-12b7-4df0-9e14-7edb227c63dc None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:41.312715 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3a139-fa09-4074-a9f5-e86b2e681443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:41.313609 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3a139-fa09-4074-a9f5-e86b2e681443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:41.314126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3a139-fa09-4074-a9f5-e86b2e681443 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:41.314586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3a139-fa09-4074-a9f5-e86b2e681443 admin admin] PATH_INFO: `/v3/projects/5e00b70b79844467a746eab7407353e3` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:41.315879 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3a139-fa09-4074-a9f5-e86b2e681443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e00b70b79844467a746eab7407353e3)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:41.322534 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3a139-fa09-4074-a9f5-e86b2e681443 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:41.382525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4aa3a139-fa09-4074-a9f5-e86b2e681443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e00b70b79844467a746eab7407353e3', 'request_id': 'req-4aa3a139-fa09-4074-a9f5-e86b2e681443'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:41.389923 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4aa3a139-fa09-4074-a9f5-e86b2e681443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e00b70b79844467a746eab7407353e3', 'request_id': 'req-4aa3a139-fa09-4074-a9f5-e86b2e681443'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:41.396475 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4aa3a139-fa09-4074-a9f5-e86b2e681443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e00b70b79844467a746eab7407353e3', 'request_id': 'req-4aa3a139-fa09-4074-a9f5-e86b2e681443'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:41.414269 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4aa3a139-fa09-4074-a9f5-e86b2e681443 admin admin] The token cache is being invalidate because project 5e00b70b79844467a746eab7407353e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:41.414562 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4aa3a139-fa09-4074-a9f5-e86b2e681443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:41.415244 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-4aa3a139-fa09-4074-a9f5-e86b2e681443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:41.416759 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1435/2886] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:41 2022] DELETE /identity/v3/projects/5e00b70b79844467a746eab7407353e3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:41.423391 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cd4e6-1fba-49d6-a008-22a6a7ae4e81 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:41.423698 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cd4e6-1fba-49d6-a008-22a6a7ae4e81 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:41.423957 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cd4e6-1fba-49d6-a008-22a6a7ae4e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:41.463681 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-dd7cd4e6-1fba-49d6-a008-22a6a7ae4e81 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:41.465774 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-dd7cd4e6-1fba-49d6-a008-22a6a7ae4e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:41.491302 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:41.542343 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1452/2887] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:41.552551 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aa046-26dc-4915-8edd-01244931b570 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:41.553396 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aa046-26dc-4915-8edd-01244931b570 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:41.553672 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54aa046-26dc-4915-8edd-01244931b570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:41.581140 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-e54aa046-26dc-4915-8edd-01244931b570 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:41.582489 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e54aa046-26dc-4915-8edd-01244931b570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:41.598268 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1436/2888] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:41.599569 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cd4e6-1fba-49d6-a008-22a6a7ae4e81 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:41.613451 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cd4e6-1fba-49d6-a008-22a6a7ae4e81 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:41.618497 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e644b7-4e2b-4ca4-8178-b25ea8850a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:41.619190 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e644b7-4e2b-4ca4-8178-b25ea8850a06 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:41.619556 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e644b7-4e2b-4ca4-8178-b25ea8850a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:41.619824 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e644b7-4e2b-4ca4-8178-b25ea8850a06 admin admin] PATH_INFO: `/v3/domains` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:41.620478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e644b7-4e2b-4ca4-8178-b25ea8850a06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87917) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:41.620780 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e644b7-4e2b-4ca4-8178-b25ea8850a06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:41.622705 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e644b7-4e2b-4ca4-8178-b25ea8850a06 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:41.637767 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1453/2889] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:03:41.642660 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54aa046-26dc-4915-8edd-01244931b570 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:41.653172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54aa046-26dc-4915-8edd-01244931b570 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:41.657826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33e53c-290a-45a9-9ec2-c315122752b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:41.658434 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33e53c-290a-45a9-9ec2-c315122752b8 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:41.658709 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33e53c-290a-45a9-9ec2-c315122752b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:41.658985 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33e53c-290a-45a9-9ec2-c315122752b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:41.659765 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33e53c-290a-45a9-9ec2-c315122752b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:41.661527 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33e53c-290a-45a9-9ec2-c315122752b8 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:41.695222 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1437/2890] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:41 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:41.699672 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e644b7-4e2b-4ca4-8178-b25ea8850a06 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:41.713846 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e644b7-4e2b-4ca4-8178-b25ea8850a06 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:41.786726 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4666533-4012-4986-94d7-e87d71d89fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:41.787341 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4666533-4012-4986-94d7-e87d71d89fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:41.787647 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4666533-4012-4986-94d7-e87d71d89fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:41.787884 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4666533-4012-4986-94d7-e87d71d89fa5 admin admin] PATH_INFO: `/v3/users` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:41.788583 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4666533-4012-4986-94d7-e87d71d89fa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:41.790527 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4666533-4012-4986-94d7-e87d71d89fa5 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:41.815349 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1454/2891] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:41 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:03:41.820470 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33e53c-290a-45a9-9ec2-c315122752b8 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:41.834631 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33e53c-290a-45a9-9ec2-c315122752b8 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:41.841624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e99a8-47f4-40a8-907a-3830b3f2408b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:41.842477 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e99a8-47f4-40a8-907a-3830b3f2408b admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:41.842850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e99a8-47f4-40a8-907a-3830b3f2408b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:41.843126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e99a8-47f4-40a8-907a-3830b3f2408b admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:41.843800 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e99a8-47f4-40a8-907a-3830b3f2408b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:41.845474 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e99a8-47f4-40a8-907a-3830b3f2408b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:41.859070 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1438/2892] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:03:41.862912 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4666533-4012-4986-94d7-e87d71d89fa5 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:41.874134 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4666533-4012-4986-94d7-e87d71d89fa5 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:41.879370 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d420b-d910-4a12-93d3-6b1aef65f7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:41.880117 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d420b-d910-4a12-93d3-6b1aef65f7d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:41.880465 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d420b-d910-4a12-93d3-6b1aef65f7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:41.880801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d420b-d910-4a12-93d3-6b1aef65f7d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:41.881535 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d420b-d910-4a12-93d3-6b1aef65f7d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:41.883829 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446d420b-d910-4a12-93d3-6b1aef65f7d1 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:41.896849 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1455/2893] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:41.901849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e99a8-47f4-40a8-907a-3830b3f2408b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:41.913390 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e99a8-47f4-40a8-907a-3830b3f2408b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:41.918404 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27568813-b1ec-4294-b76c-8984eb53b8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:41.919095 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27568813-b1ec-4294-b76c-8984eb53b8a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:41.919372 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27568813-b1ec-4294-b76c-8984eb53b8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:41.919646 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27568813-b1ec-4294-b76c-8984eb53b8a0 admin admin] PATH_INFO: `/v3/projects/79fb8584dccb42ff85a789fb304707d4/users/d110f0e7aa9246a18c456970a1241471/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:41.943693 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27568813-b1ec-4294-b76c-8984eb53b8a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79fb8584dccb42ff85a789fb304707d4, user_id=d110f0e7aa9246a18c456970a1241471, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:41.947329 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27568813-b1ec-4294-b76c-8984eb53b8a0 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:41.968379 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1439/2894] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:41 2022] PUT /identity/v3/projects/79fb8584dccb42ff85a789fb304707d4/users/d110f0e7aa9246a18c456970a1241471/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:41.974177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274e86e-1654-4560-ba03-4249e39cab15 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:41.974492 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274e86e-1654-4560-ba03-4249e39cab15 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:41.974748 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274e86e-1654-4560-ba03-4249e39cab15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:42.027915 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-b274e86e-1654-4560-ba03-4249e39cab15 None None] MFA Rules not processed for user `d110f0e7aa9246a18c456970a1241471`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:42.029142 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-b274e86e-1654-4560-ba03-4249e39cab15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:42.109280 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1456/2895] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:03:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:42.123718 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27568813-b1ec-4294-b76c-8984eb53b8a0 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:42.125624 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-27568813-b1ec-4294-b76c-8984eb53b8a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:42.217435 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27568813-b1ec-4294-b76c-8984eb53b8a0 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:42.221456 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16b5c4-d02c-4b83-9d11-39a7eaa15971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:42.222139 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16b5c4-d02c-4b83-9d11-39a7eaa15971 service cinder] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:42.222471 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16b5c4-d02c-4b83-9d11-39a7eaa15971 service cinder] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:42.222751 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16b5c4-d02c-4b83-9d11-39a7eaa15971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:42.256438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16b5c4-d02c-4b83-9d11-39a7eaa15971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:42.260950 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16b5c4-d02c-4b83-9d11-39a7eaa15971 service cinder] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:42.273712 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1440/2896] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:03:42.302029 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274e86e-1654-4560-ba03-4249e39cab15 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:42.327983 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:42.388378 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274e86e-1654-4560-ba03-4249e39cab15 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:42.393545 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:42.394453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:42.395870 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:42.396325 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] PATH_INFO: `/v3/users/d110f0e7aa9246a18c456970a1241471` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:42.397551 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d110f0e7aa9246a18c456970a1241471)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:42.399566 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:42.487403 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd110f0e7aa9246a18c456970a1241471'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:42.506313 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd110f0e7aa9246a18c456970a1241471'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:42.509039 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd110f0e7aa9246a18c456970a1241471'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:42.510049 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd110f0e7aa9246a18c456970a1241471'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:42.521769 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd110f0e7aa9246a18c456970a1241471'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:42.549054 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1457/2897] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:42 2022] DELETE /identity/v3/users/d110f0e7aa9246a18c456970a1241471 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:42.553710 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d24846-c0d9-4fbc-8d75-401c79ad9a18 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:42.554172 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d24846-c0d9-4fbc-8d75-401c79ad9a18 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:42.554554 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d24846-c0d9-4fbc-8d75-401c79ad9a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:42.576616 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-87d24846-c0d9-4fbc-8d75-401c79ad9a18 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:42.578000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-87d24846-c0d9-4fbc-8d75-401c79ad9a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:42.584131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:42.662885 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1441/2898] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:42.672973 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:42.675127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:42.748234 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58b68d-bd67-4aa3-8cd9-11a8811231c4 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:42.756801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba812c3-f36b-4477-95de-415c380c2c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:42.757536 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba812c3-f36b-4477-95de-415c380c2c9d service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:42.758424 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba812c3-f36b-4477-95de-415c380c2c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:42.759188 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba812c3-f36b-4477-95de-415c380c2c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:42.779671 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba812c3-f36b-4477-95de-415c380c2c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:42.782175 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba812c3-f36b-4477-95de-415c380c2c9d service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:42.802124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1458/2899] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:43.330245 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d24846-c0d9-4fbc-8d75-401c79ad9a18 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:43.332771 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-87d24846-c0d9-4fbc-8d75-401c79ad9a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:43.357853 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d24846-c0d9-4fbc-8d75-401c79ad9a18 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:43.365084 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:43.366126 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:43.377303 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:43.377711 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] PATH_INFO: `/v3/projects/79fb8584dccb42ff85a789fb304707d4` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:43.379231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] RBAC: Authorizing `identity:delete_project(project_id=79fb8584dccb42ff85a789fb304707d4)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:43.381356 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:43.476355 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79fb8584dccb42ff85a789fb304707d4', 'request_id': 'req-c29455b3-1dab-4ef1-b817-39b41991693d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:43.485501 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79fb8584dccb42ff85a789fb304707d4', 'request_id': 'req-c29455b3-1dab-4ef1-b817-39b41991693d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:43.493584 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79fb8584dccb42ff85a789fb304707d4', 'request_id': 'req-c29455b3-1dab-4ef1-b817-39b41991693d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:43.511996 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] The token cache is being invalidate because project 79fb8584dccb42ff85a789fb304707d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87916) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:43.512288 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:43.514297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:43.516091 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1442/2900] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:43 2022] DELETE /identity/v3/projects/79fb8584dccb42ff85a789fb304707d4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:48.051723 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba812c3-f36b-4477-95de-415c380c2c9d service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:48.054381 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-7ba812c3-f36b-4477-95de-415c380c2c9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:48.067956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba812c3-f36b-4477-95de-415c380c2c9d service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:48.123177 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:48.146118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcdaea-4fe0-4359-b007-742b73cf934b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:48.147005 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcdaea-4fe0-4359-b007-742b73cf934b service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:48.147414 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcdaea-4fe0-4359-b007-742b73cf934b service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:48.147810 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcdaea-4fe0-4359-b007-742b73cf934b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:48.149962 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-3ddcdaea-4fe0-4359-b007-742b73cf934b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:48.236001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddcdaea-4fe0-4359-b007-742b73cf934b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:48.237619 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddcdaea-4fe0-4359-b007-742b73cf934b service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:48.251024 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1459/2901] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:50.237774 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:50.239439 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:50.345680 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29455b3-1dab-4ef1-b817-39b41991693d admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:50.353570 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83281c85-a061-4d0d-8c22-02ee4e3e5dbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:50.355339 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83281c85-a061-4d0d-8c22-02ee4e3e5dbd service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:50.356366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83281c85-a061-4d0d-8c22-02ee4e3e5dbd service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:50.357152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83281c85-a061-4d0d-8c22-02ee4e3e5dbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:50.360736 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-83281c85-a061-4d0d-8c22-02ee4e3e5dbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:50.380504 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83281c85-a061-4d0d-8c22-02ee4e3e5dbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:50.383392 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83281c85-a061-4d0d-8c22-02ee4e3e5dbd service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:50.409622 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1443/2902] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:03:53.422298 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84808c6c-0334-4b18-bbad-db44954d637b None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:53.422774 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84808c6c-0334-4b18-bbad-db44954d637b None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:53.422956 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84808c6c-0334-4b18-bbad-db44954d637b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:53.442744 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-84808c6c-0334-4b18-bbad-db44954d637b None None] MFA Rules not processed for user `41a5f117527a41fca00978c35fbd9eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:53.443941 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-84808c6c-0334-4b18-bbad-db44954d637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:53.498689 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1460/2903] 10.222.0.55 () {58 vars in 967 bytes} [Thu Sep 8 14:03:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:53.511553 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83281c85-a061-4d0d-8c22-02ee4e3e5dbd service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:53.512932 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-83281c85-a061-4d0d-8c22-02ee4e3e5dbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:53.534148 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:53.580994 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83281c85-a061-4d0d-8c22-02ee4e3e5dbd service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:53.585892 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24845835-2f9a-48e3-8cd6-afb430f4be42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e5d73e8daa84975bc21e41e3cbef9cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:53.586489 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24845835-2f9a-48e3-8cd6-afb430f4be42 service swift] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:53.586764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24845835-2f9a-48e3-8cd6-afb430f4be42 service swift] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:53.587030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24845835-2f9a-48e3-8cd6-afb430f4be42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:53.595284 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24845835-2f9a-48e3-8cd6-afb430f4be42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:53.596849 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24845835-2f9a-48e3-8cd6-afb430f4be42 service swift] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:53.605368 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1444/2904] 10.222.0.55 () {60 vars in 1384 bytes} [Thu Sep 8 14:03:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:03:55.446124 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84808c6c-0334-4b18-bbad-db44954d637b None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:55.447573 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-84808c6c-0334-4b18-bbad-db44954d637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:55.501673 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24845835-2f9a-48e3-8cd6-afb430f4be42 service swift] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:55.503270 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-24845835-2f9a-48e3-8cd6-afb430f4be42 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:55.523335 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84808c6c-0334-4b18-bbad-db44954d637b None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:55.528547 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351090a-8f77-4cc0-b2c5-35924a471a2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:55.529162 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351090a-8f77-4cc0-b2c5-35924a471a2f service placement] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:55.529446 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351090a-8f77-4cc0-b2c5-35924a471a2f service placement] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:55.529719 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351090a-8f77-4cc0-b2c5-35924a471a2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:55.531544 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-0351090a-8f77-4cc0-b2c5-35924a471a2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:55.544713 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351090a-8f77-4cc0-b2c5-35924a471a2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:55.546360 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351090a-8f77-4cc0-b2c5-35924a471a2f service placement] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:55.557097 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1461/2905] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:55.572315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24845835-2f9a-48e3-8cd6-afb430f4be42 service swift] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:55.579660 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c930d-82bd-48f9-8033-3b038d616fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:55.580083 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c930d-82bd-48f9-8033-3b038d616fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:55.580366 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c930d-82bd-48f9-8033-3b038d616fee admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:55.580644 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c930d-82bd-48f9-8033-3b038d616fee admin admin] PATH_INFO: `/v3/users/52bda0133af44d5ca220ba869fbcb3a9` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:55.581878 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c930d-82bd-48f9-8033-3b038d616fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=52bda0133af44d5ca220ba869fbcb3a9)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:55.583896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c930d-82bd-48f9-8033-3b038d616fee admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:55.640259 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f79c930d-82bd-48f9-8033-3b038d616fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52bda0133af44d5ca220ba869fbcb3a9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:55.651793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f79c930d-82bd-48f9-8033-3b038d616fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52bda0133af44d5ca220ba869fbcb3a9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:55.652409 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f79c930d-82bd-48f9-8033-3b038d616fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52bda0133af44d5ca220ba869fbcb3a9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:55.653102 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f79c930d-82bd-48f9-8033-3b038d616fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52bda0133af44d5ca220ba869fbcb3a9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:55.660081 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-f79c930d-82bd-48f9-8033-3b038d616fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52bda0133af44d5ca220ba869fbcb3a9'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:55.677333 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1445/2906] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:55 2022] DELETE /identity/v3/users/52bda0133af44d5ca220ba869fbcb3a9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:55.851808 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351090a-8f77-4cc0-b2c5-35924a471a2f service placement] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:55.863841 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351090a-8f77-4cc0-b2c5-35924a471a2f service placement] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:55.894072 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=100 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:55.926965 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c3c943-1933-4da0-ad5d-9777bc790901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:55.927597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3c943-1933-4da0-ad5d-9777bc790901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:55.927877 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3c943-1933-4da0-ad5d-9777bc790901 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:55.928155 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3c943-1933-4da0-ad5d-9777bc790901 admin admin] PATH_INFO: `/v3/projects/6a848414c86b439c922442d0862c334f` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:55.929276 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3c943-1933-4da0-ad5d-9777bc790901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a848414c86b439c922442d0862c334f)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:55.931211 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c3c943-1933-4da0-ad5d-9777bc790901 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:55.976979 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a5c3c943-1933-4da0-ad5d-9777bc790901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a848414c86b439c922442d0862c334f', 'request_id': 'req-a5c3c943-1933-4da0-ad5d-9777bc790901'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:55.983118 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a5c3c943-1933-4da0-ad5d-9777bc790901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a848414c86b439c922442d0862c334f', 'request_id': 'req-a5c3c943-1933-4da0-ad5d-9777bc790901'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:55.990316 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a5c3c943-1933-4da0-ad5d-9777bc790901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a848414c86b439c922442d0862c334f', 'request_id': 'req-a5c3c943-1933-4da0-ad5d-9777bc790901'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:56.000705 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a5c3c943-1933-4da0-ad5d-9777bc790901 admin admin] The token cache is being invalidate because project 6a848414c86b439c922442d0862c334f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:56.000993 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a5c3c943-1933-4da0-ad5d-9777bc790901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:56.001618 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-a5c3c943-1933-4da0-ad5d-9777bc790901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:56.003107 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1462/2907] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:55 2022] DELETE /identity/v3/projects/6a848414c86b439c922442d0862c334f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:56.008530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604263b4-521f-46e8-afdc-922f55ae449e None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:56.008826 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604263b4-521f-46e8-afdc-922f55ae449e None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:56.009101 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604263b4-521f-46e8-afdc-922f55ae449e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:56.028791 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-604263b4-521f-46e8-afdc-922f55ae449e None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:56.030023 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-604263b4-521f-46e8-afdc-922f55ae449e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:56.053628 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:56.091857 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1446/2908] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:56.099464 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539405b-d4a3-4fd4-9804-9c96d91bfb96 None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:56.099783 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539405b-d4a3-4fd4-9804-9c96d91bfb96 None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:56.100057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539405b-d4a3-4fd4-9804-9c96d91bfb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:56.119236 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-9539405b-d4a3-4fd4-9804-9c96d91bfb96 None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:56.120416 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-9539405b-d4a3-4fd4-9804-9c96d91bfb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:56.128125 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1463/2909] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:56.132795 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604263b4-521f-46e8-afdc-922f55ae449e None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:56.142541 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604263b4-521f-46e8-afdc-922f55ae449e None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:56.147250 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc695e9-ef38-4364-9671-0fbf6b39de56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:56.148444 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc695e9-ef38-4364-9671-0fbf6b39de56 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:56.148716 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc695e9-ef38-4364-9671-0fbf6b39de56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:56.148987 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc695e9-ef38-4364-9671-0fbf6b39de56 admin admin] PATH_INFO: `/v3/domains` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:56.149634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc695e9-ef38-4364-9671-0fbf6b39de56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87916) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:56.149972 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc695e9-ef38-4364-9671-0fbf6b39de56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:56.151586 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc695e9-ef38-4364-9671-0fbf6b39de56 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:56.163037 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1447/2910] 10.222.0.55 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:03:56.167432 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539405b-d4a3-4fd4-9804-9c96d91bfb96 None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:56.176452 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539405b-d4a3-4fd4-9804-9c96d91bfb96 None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:56.181106 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486f6c0-8b73-4447-9c1d-1b0040fd3afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:56.181691 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486f6c0-8b73-4447-9c1d-1b0040fd3afc admin admin] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:56.182114 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486f6c0-8b73-4447-9c1d-1b0040fd3afc admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:56.182383 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486f6c0-8b73-4447-9c1d-1b0040fd3afc admin admin] PATH_INFO: `/v3/projects` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:56.183010 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486f6c0-8b73-4447-9c1d-1b0040fd3afc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:56.184877 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486f6c0-8b73-4447-9c1d-1b0040fd3afc admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:56.214502 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1464/2911] 10.222.0.55 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:56 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:56.218848 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc695e9-ef38-4364-9671-0fbf6b39de56 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:56.228861 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc695e9-ef38-4364-9671-0fbf6b39de56 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:56.285097 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86530b7d-469d-44f6-b457-43d19b696af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:56.285706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86530b7d-469d-44f6-b457-43d19b696af9 admin admin] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:56.286092 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86530b7d-469d-44f6-b457-43d19b696af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:56.286370 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86530b7d-469d-44f6-b457-43d19b696af9 admin admin] PATH_INFO: `/v3/users` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:56.287000 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86530b7d-469d-44f6-b457-43d19b696af9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:56.288953 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86530b7d-469d-44f6-b457-43d19b696af9 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:56.310029 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1448/2912] 10.222.0.55 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:56 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:56.314150 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486f6c0-8b73-4447-9c1d-1b0040fd3afc admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:56.325230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486f6c0-8b73-4447-9c1d-1b0040fd3afc admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:56.329899 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3653c9-3561-4b08-a48d-b9014f9817fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:56.330516 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3653c9-3561-4b08-a48d-b9014f9817fb admin admin] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:56.330794 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3653c9-3561-4b08-a48d-b9014f9817fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:56.331063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3653c9-3561-4b08-a48d-b9014f9817fb admin admin] PATH_INFO: `/v3/roles` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:56.331701 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3653c9-3561-4b08-a48d-b9014f9817fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:56.333277 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3653c9-3561-4b08-a48d-b9014f9817fb admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:56.343574 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1465/2913] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:56.347376 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86530b7d-469d-44f6-b457-43d19b696af9 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:56.357425 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86530b7d-469d-44f6-b457-43d19b696af9 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:56.362166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760832e-f8f0-4c35-8e70-d705d0d95296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:56.362763 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760832e-f8f0-4c35-8e70-d705d0d95296 admin admin] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:56.363047 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760832e-f8f0-4c35-8e70-d705d0d95296 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:56.363325 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760832e-f8f0-4c35-8e70-d705d0d95296 admin admin] PATH_INFO: `/v3/roles` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:56.363966 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760832e-f8f0-4c35-8e70-d705d0d95296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:56.365587 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760832e-f8f0-4c35-8e70-d705d0d95296 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:56.376656 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1449/2914] 10.222.0.55 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:56.380595 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3653c9-3561-4b08-a48d-b9014f9817fb admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:56.390986 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3653c9-3561-4b08-a48d-b9014f9817fb admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:56.395543 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0ba8e-f032-4d69-b888-a2cb6159b5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:56.396176 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0ba8e-f032-4d69-b888-a2cb6159b5b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:56.396453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0ba8e-f032-4d69-b888-a2cb6159b5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:56.396725 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc0ba8e-f032-4d69-b888-a2cb6159b5b9 admin admin] PATH_INFO: `/v3/projects/3649c904f2d9429ba62ca500420838ec/users/0713ae5eeb96458a95bb4cefa4daeae7/roles/8abe8932c5974e7db6fb974c588a9d70` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:56.411218 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0ba8e-f032-4d69-b888-a2cb6159b5b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3649c904f2d9429ba62ca500420838ec, user_id=0713ae5eeb96458a95bb4cefa4daeae7, role_id=8abe8932c5974e7db6fb974c588a9d70)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:56.413967 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc0ba8e-f032-4d69-b888-a2cb6159b5b9 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:56.427801 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1466/2915] 10.222.0.55 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:56 2022] PUT /identity/v3/projects/3649c904f2d9429ba62ca500420838ec/users/0713ae5eeb96458a95bb4cefa4daeae7/roles/8abe8932c5974e7db6fb974c588a9d70 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:56.439346 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dfc85-8b3f-40c4-87ac-1fa2e68112c2 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:56.439637 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dfc85-8b3f-40c4-87ac-1fa2e68112c2 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:56.439938 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dfc85-8b3f-40c4-87ac-1fa2e68112c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:56.467876 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b76dfc85-8b3f-40c4-87ac-1fa2e68112c2 None None] MFA Rules not processed for user `0713ae5eeb96458a95bb4cefa4daeae7`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:56.469096 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b76dfc85-8b3f-40c4-87ac-1fa2e68112c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:56.535177 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1450/2916] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:03:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:56.546001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0ba8e-f032-4d69-b888-a2cb6159b5b9 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:56.547775 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-3fc0ba8e-f032-4d69-b888-a2cb6159b5b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:56.605256 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc0ba8e-f032-4d69-b888-a2cb6159b5b9 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:56.608889 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c7a90-cb24-4dbe-8850-e3f76c77dab6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff93243f0c4c48d1924ea04a87a0e7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:56.609486 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c7a90-cb24-4dbe-8850-e3f76c77dab6 service cinder] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:56.609792 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c7a90-cb24-4dbe-8850-e3f76c77dab6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:56.610074 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c7a90-cb24-4dbe-8850-e3f76c77dab6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:56.616230 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:56.619306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c7a90-cb24-4dbe-8850-e3f76c77dab6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:56.621033 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c7a90-cb24-4dbe-8850-e3f76c77dab6 service cinder] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:56.631397 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1467/2917] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:06.532308 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:06.632191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:06.778652 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dfc85-8b3f-40c4-87ac-1fa2e68112c2 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:06.839367 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dfc85-8b3f-40c4-87ac-1fa2e68112c2 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:06.844087 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:06.844706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:06.844986 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:06.845257 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] PATH_INFO: `/v3/users/0713ae5eeb96458a95bb4cefa4daeae7` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:06.846407 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0713ae5eeb96458a95bb4cefa4daeae7)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:06.848427 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:06.893472 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0713ae5eeb96458a95bb4cefa4daeae7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:06.901449 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0713ae5eeb96458a95bb4cefa4daeae7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:06.902219 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0713ae5eeb96458a95bb4cefa4daeae7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:06.902893 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0713ae5eeb96458a95bb4cefa4daeae7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:06.910499 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0713ae5eeb96458a95bb4cefa4daeae7'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:06.927690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1451/2918] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:04:06 2022] DELETE /identity/v3/users/0713ae5eeb96458a95bb4cefa4daeae7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:06.932165 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e70681-898e-4794-be39-906e66b8895d None None] REQUEST_METHOD: `POST` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:06.932453 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e70681-898e-4794-be39-906e66b8895d None None] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:06.932736 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e70681-898e-4794-be39-906e66b8895d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:06.951708 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.auth.core [None req-d7e70681-898e-4794-be39-906e66b8895d None None] MFA Rules not processed for user `3709ff21e04749f7aae9073ca845e875`. Rule list: `[]` (Enabled: `True`). {{(pid=87917) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:06.952938 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d7e70681-898e-4794-be39-906e66b8895d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:07.011291 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1468/2919] 10.222.0.55 () {58 vars in 958 bytes} [Thu Sep 8 14:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:07.020784 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:07.022721 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:07.079914 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84522371-d94b-4d74-8aea-4cc343d5cb1b admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:07.083539 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b5493-7fea-4ac9-8c11-6eaf39e3b1eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:07.084136 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b5493-7fea-4ac9-8c11-6eaf39e3b1eb service neutron] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:07.084416 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b5493-7fea-4ac9-8c11-6eaf39e3b1eb service neutron] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:07.084764 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b5493-7fea-4ac9-8c11-6eaf39e3b1eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:07.093808 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b5493-7fea-4ac9-8c11-6eaf39e3b1eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:07.095202 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b5493-7fea-4ac9-8c11-6eaf39e3b1eb service neutron] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:07.105099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1452/2920] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:07.364677 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e70681-898e-4794-be39-906e66b8895d None None] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:07.367057 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-d7e70681-898e-4794-be39-906e66b8895d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:07.379148 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e70681-898e-4794-be39-906e66b8895d None None] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:07.383817 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:07.384446 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:07.384724 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:07.384992 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] PATH_INFO: `/v3/projects/3649c904f2d9429ba62ca500420838ec` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:07.386142 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3649c904f2d9429ba62ca500420838ec)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:07.388043 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:07.434265 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3649c904f2d9429ba62ca500420838ec', 'request_id': 'req-3124161b-2582-4f54-9373-c4794cab38cd'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:07.440469 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3649c904f2d9429ba62ca500420838ec', 'request_id': 'req-3124161b-2582-4f54-9373-c4794cab38cd'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:07.447405 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3649c904f2d9429ba62ca500420838ec', 'request_id': 'req-3124161b-2582-4f54-9373-c4794cab38cd'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:07.457876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] The token cache is being invalidate because project 3649c904f2d9429ba62ca500420838ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:04:07.458165 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:07.458804 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:07.460823 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1469/2921] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:04:07 2022] DELETE /identity/v3/projects/3649c904f2d9429ba62ca500420838ec => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:17.100481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:17.454717 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:41.654156 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b5493-7fea-4ac9-8c11-6eaf39e3b1eb service neutron] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:41.655677 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3a8b5493-7fea-4ac9-8c11-6eaf39e3b1eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:41.725905 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b5493-7fea-4ac9-8c11-6eaf39e3b1eb service neutron] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:41.783706 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f213f-b6eb-42d3-b5e3-af84f6cd7c46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:41.784307 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f213f-b6eb-42d3-b5e3-af84f6cd7c46 service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:41.784582 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f213f-b6eb-42d3-b5e3-af84f6cd7c46 service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:41.784850 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f213f-b6eb-42d3-b5e3-af84f6cd7c46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:41.786580 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-786f213f-b6eb-42d3-b5e3-af84f6cd7c46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:41.797870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f213f-b6eb-42d3-b5e3-af84f6cd7c46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:41.799421 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f213f-b6eb-42d3-b5e3-af84f6cd7c46 service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:41.809039 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1453/2922] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:46.269183 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:46.270847 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:46.329712 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3124161b-2582-4f54-9373-c4794cab38cd admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:46.333871 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cc54e-f151-4ab6-b3c0-cbd4378e4d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:46.335075 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cc54e-f151-4ab6-b3c0-cbd4378e4d98 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:46.335417 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cc54e-f151-4ab6-b3c0-cbd4378e4d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:46.335752 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cc54e-f151-4ab6-b3c0-cbd4378e4d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:46.337514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-681cc54e-f151-4ab6-b3c0-cbd4378e4d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:46.402718 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cc54e-f151-4ab6-b3c0-cbd4378e4d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:46.404159 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cc54e-f151-4ab6-b3c0-cbd4378e4d98 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:46.415366 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1470/2923] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:49.522180 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f213f-b6eb-42d3-b5e3-af84f6cd7c46 service placement] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:49.523720 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-786f213f-b6eb-42d3-b5e3-af84f6cd7c46 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:49.543793 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f213f-b6eb-42d3-b5e3-af84f6cd7c46 service placement] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:49.549526 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e824f-0e31-4ccb-a95b-8c6c48d970c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:49.550231 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e824f-0e31-4ccb-a95b-8c6c48d970c2 service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:49.550512 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e824f-0e31-4ccb-a95b-8c6c48d970c2 service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:49.550786 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e824f-0e31-4ccb-a95b-8c6c48d970c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:49.552530 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e72e824f-0e31-4ccb-a95b-8c6c48d970c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:49.566208 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e824f-0e31-4ccb-a95b-8c6c48d970c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:49.567811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e824f-0e31-4ccb-a95b-8c6c48d970c2 service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:49.584844 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1454/2924] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:52.549542 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cc54e-f151-4ab6-b3c0-cbd4378e4d98 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:52.551918 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-681cc54e-f151-4ab6-b3c0-cbd4378e4d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:52.571478 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cc54e-f151-4ab6-b3c0-cbd4378e4d98 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:52.576062 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49666b4-2d28-43cb-a178-8ed15907d5f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:52.577501 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49666b4-2d28-43cb-a178-8ed15907d5f9 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:52.578359 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49666b4-2d28-43cb-a178-8ed15907d5f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:52.579064 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49666b4-2d28-43cb-a178-8ed15907d5f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:52.581721 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-f49666b4-2d28-43cb-a178-8ed15907d5f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:52.610607 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49666b4-2d28-43cb-a178-8ed15907d5f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:52.613491 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49666b4-2d28-43cb-a178-8ed15907d5f9 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:52.624484 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1471/2925] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:52.917829 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e824f-0e31-4ccb-a95b-8c6c48d970c2 service placement] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:52.919438 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-e72e824f-0e31-4ccb-a95b-8c6c48d970c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:52.934731 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e824f-0e31-4ccb-a95b-8c6c48d970c2 service placement] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:52.939195 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:52.939821 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:52.940103 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:52.940380 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] PATH_INFO: `/v3/users/76a9dc3ddb144e599a2856fce8dc339d` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:52.941500 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76a9dc3ddb144e599a2856fce8dc339d)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:52.944487 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:52.992745 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76a9dc3ddb144e599a2856fce8dc339d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.000315 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76a9dc3ddb144e599a2856fce8dc339d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.001167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76a9dc3ddb144e599a2856fce8dc339d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.001822 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76a9dc3ddb144e599a2856fce8dc339d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.009332 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76a9dc3ddb144e599a2856fce8dc339d'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.016998 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:53.028528 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1455/2926] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:05:52 2022] DELETE /identity/v3/users/76a9dc3ddb144e599a2856fce8dc339d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:05:53.196574 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49666b4-2d28-43cb-a178-8ed15907d5f9 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:53.208514 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49666b4-2d28-43cb-a178-8ed15907d5f9 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:53.266232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:53.267083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:53.267363 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:53.267635 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] PATH_INFO: `/v3/projects/19d114fd258a4306b4c0665df90ae4c7` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:53.268765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d114fd258a4306b4c0665df90ae4c7)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:53.270707 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:53.314905 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:53.323253 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d114fd258a4306b4c0665df90ae4c7', 'request_id': 'req-633580af-9086-42d3-9fac-5bc094f01364'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.329482 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d114fd258a4306b4c0665df90ae4c7', 'request_id': 'req-633580af-9086-42d3-9fac-5bc094f01364'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.336855 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d114fd258a4306b4c0665df90ae4c7', 'request_id': 'req-633580af-9086-42d3-9fac-5bc094f01364'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.347639 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] The token cache is being invalidate because project 19d114fd258a4306b4c0665df90ae4c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:05:53.347930 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.348552 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.350582 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1472/2927] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:05:53 2022] DELETE /identity/v3/projects/19d114fd258a4306b4c0665df90ae4c7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:05:53.355165 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:53.356592 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:53.370064 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22789ccd-5428-4351-aeea-a44786e9bc71 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:53.425153 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5f629-2212-462c-a4d1-c976f12b35e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:53.425811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5f629-2212-462c-a4d1-c976f12b35e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:53.426093 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5f629-2212-462c-a4d1-c976f12b35e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:53.426369 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5f629-2212-462c-a4d1-c976f12b35e6 admin admin] PATH_INFO: `/v3/users/b30ed315686e4801b490d5d658de5d7f` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:53.427507 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5f629-2212-462c-a4d1-c976f12b35e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30ed315686e4801b490d5d658de5d7f)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:53.430166 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5f629-2212-462c-a4d1-c976f12b35e6 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:53.478559 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9aa5f629-2212-462c-a4d1-c976f12b35e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30ed315686e4801b490d5d658de5d7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.485870 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9aa5f629-2212-462c-a4d1-c976f12b35e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30ed315686e4801b490d5d658de5d7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.486522 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9aa5f629-2212-462c-a4d1-c976f12b35e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30ed315686e4801b490d5d658de5d7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.487122 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9aa5f629-2212-462c-a4d1-c976f12b35e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30ed315686e4801b490d5d658de5d7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.494689 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-9aa5f629-2212-462c-a4d1-c976f12b35e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30ed315686e4801b490d5d658de5d7f'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.511552 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1456/2928] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:05:53 2022] DELETE /identity/v3/users/b30ed315686e4801b490d5d658de5d7f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:05:53.688795 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:53.690367 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:53.706001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633580af-9086-42d3-9fac-5bc094f01364 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:53.763150 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:53.763788 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:53.764068 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:53.764337 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] PATH_INFO: `/v3/projects/bd737a2dedef4e1cb1ea490cd45a985d` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:53.766093 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd737a2dedef4e1cb1ea490cd45a985d)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:53.767505 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:53.776811 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134758-2ada-40cc-891d-e4e9e2e82fe9 None None] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:53.777167 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134758-2ada-40cc-891d-e4e9e2e82fe9 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:53.777450 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134758-2ada-40cc-891d-e4e9e2e82fe9 None None] PATH_INFO: `/` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:53.778980 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1457/2929] 10.222.0.55 () {54 vars in 921 bytes} [Thu Sep 8 14:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 14:05:53.783434 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e253db-aef3-4c49-ba7b-df7d3efc8405 None None] REQUEST_METHOD: `POST` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:53.783787 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e253db-aef3-4c49-ba7b-df7d3efc8405 None None] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:53.784058 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e253db-aef3-4c49-ba7b-df7d3efc8405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:53.816413 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd737a2dedef4e1cb1ea490cd45a985d', 'request_id': 'req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.822655 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd737a2dedef4e1cb1ea490cd45a985d', 'request_id': 'req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.823614 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.auth.core [None req-b2e253db-aef3-4c49-ba7b-df7d3efc8405 None None] MFA Rules not processed for user `6ffa73f81539472391983c51447af1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=87916) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:53.824834 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b2e253db-aef3-4c49-ba7b-df7d3efc8405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:53.830866 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd737a2dedef4e1cb1ea490cd45a985d', 'request_id': 'req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.844063 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] The token cache is being invalidate because project bd737a2dedef4e1cb1ea490cd45a985d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:05:53.844199 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.844838 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:53.857763 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1473/2930] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:05:53 2022] DELETE /identity/v3/projects/bd737a2dedef4e1cb1ea490cd45a985d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:05:53.894653 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:53.930544 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1458/2931] 10.222.0.55 () {58 vars in 1018 bytes} [Thu Sep 8 14:05:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:05:53.931758 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:53.933270 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:54.003320 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd64c8-fc9b-4ce9-b20a-2bd3f5c1b794 admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:54.007694 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fc52b-40ff-4379-9ec4-20b6a4b26af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:54.008451 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc52b-40ff-4379-9ec4-20b6a4b26af0 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:54.008727 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc52b-40ff-4379-9ec4-20b6a4b26af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:54.008999 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc52b-40ff-4379-9ec4-20b6a4b26af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:54.020934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fc52b-40ff-4379-9ec4-20b6a4b26af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:54.022949 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fc52b-40ff-4379-9ec4-20b6a4b26af0 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:54.030306 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:54.036065 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1474/2932] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:05:54.633442 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e253db-aef3-4c49-ba7b-df7d3efc8405 None None] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:54.634937 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b2e253db-aef3-4c49-ba7b-df7d3efc8405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:54.649225 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e253db-aef3-4c49-ba7b-df7d3efc8405 None None] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:54.654804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bd8bb-dac8-4d90-bae8-62bb21395fd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ffa73f81539472391983c51447af1b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:54.655412 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bd8bb-dac8-4d90-bae8-62bb21395fd5 service nova] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:54.655690 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bd8bb-dac8-4d90-bae8-62bb21395fd5 service nova] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:54.655964 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bd8bb-dac8-4d90-bae8-62bb21395fd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:54.657665 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-826bd8bb-dac8-4d90-bae8-62bb21395fd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:54.670768 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bd8bb-dac8-4d90-bae8-62bb21395fd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:54.672403 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826bd8bb-dac8-4d90-bae8-62bb21395fd5 service nova] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:54.683831 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1459/2933] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:04.679135 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=11 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:30.958765 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fc52b-40ff-4379-9ec4-20b6a4b26af0 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:30.960207 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-549fc52b-40ff-4379-9ec4-20b6a4b26af0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:30.972326 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fc52b-40ff-4379-9ec4-20b6a4b26af0 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:30.976053 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7027b1-e7ab-4034-9a52-01502fbe8e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '531ebc79bf424e9b8762b2c7ec7fe8d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:30.976644 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7027b1-e7ab-4034-9a52-01502fbe8e93 service neutron] REQUEST_METHOD: `GET` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:30.976920 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7027b1-e7ab-4034-9a52-01502fbe8e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:30.977191 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7027b1-e7ab-4034-9a52-01502fbe8e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:30.979464 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-4c7027b1-e7ab-4034-9a52-01502fbe8e93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:31.043001 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7027b1-e7ab-4034-9a52-01502fbe8e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:31.044421 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7027b1-e7ab-4034-9a52-01502fbe8e93 service neutron] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:31.053922 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1475/2934] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:37.237042 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bd8bb-dac8-4d90-bae8-62bb21395fd5 service nova] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:37.239152 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-826bd8bb-dac8-4d90-bae8-62bb21395fd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:37.252979 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826bd8bb-dac8-4d90-bae8-62bb21395fd5 service nova] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:37.257525 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:37.258197 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:37.258480 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:37.258757 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] PATH_INFO: `/v3/users/39393f5448ed4c489ae1459d251f50d0` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:37.259872 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39393f5448ed4c489ae1459d251f50d0)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:37.261827 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:37.308003 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39393f5448ed4c489ae1459d251f50d0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.317295 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39393f5448ed4c489ae1459d251f50d0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.318056 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39393f5448ed4c489ae1459d251f50d0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.318697 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39393f5448ed4c489ae1459d251f50d0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.327344 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39393f5448ed4c489ae1459d251f50d0'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.345099 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1460/2935] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:07:37 2022] DELETE /identity/v3/users/39393f5448ed4c489ae1459d251f50d0 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:37.517415 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7027b1-e7ab-4034-9a52-01502fbe8e93 service neutron] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:37.528597 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7027b1-e7ab-4034-9a52-01502fbe8e93 service neutron] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:37.587811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:37.588508 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:37.588834 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:37.589187 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] PATH_INFO: `/v3/projects/1dcae1b090af440f836b18a7bb000c27` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:37.590633 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dcae1b090af440f836b18a7bb000c27)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:37.592804 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:37.638919 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dcae1b090af440f836b18a7bb000c27', 'request_id': 'req-8a44815d-1c37-4e75-b94f-b114817ad66b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.645512 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dcae1b090af440f836b18a7bb000c27', 'request_id': 'req-8a44815d-1c37-4e75-b94f-b114817ad66b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.654051 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dcae1b090af440f836b18a7bb000c27', 'request_id': 'req-8a44815d-1c37-4e75-b94f-b114817ad66b'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.664636 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] The token cache is being invalidate because project 1dcae1b090af440f836b18a7bb000c27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:07:37.664934 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.665593 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.667152 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1476/2936] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:07:37 2022] DELETE /identity/v3/projects/1dcae1b090af440f836b18a7bb000c27 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:37.671310 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:37.672804 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:37.685896 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b4048-7990-43a8-8caa-bac430d8c219 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:37.746283 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:37.746837 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:37.747131 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:37.747413 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] PATH_INFO: `/v3/users/a605f837a1004c1f98d8246af920543e` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:37.748581 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a605f837a1004c1f98d8246af920543e)` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:37.750546 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:37.797282 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a605f837a1004c1f98d8246af920543e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.804681 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a605f837a1004c1f98d8246af920543e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.805306 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a605f837a1004c1f98d8246af920543e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.806481 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a605f837a1004c1f98d8246af920543e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.813360 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.notifications [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a605f837a1004c1f98d8246af920543e'} {{(pid=87916) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.821260 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:37.832511 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1461/2937] 10.222.0.55 () {58 vars in 1224 bytes} [Thu Sep 8 14:07:37 2022] DELETE /identity/v3/users/a605f837a1004c1f98d8246af920543e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:38.010315 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] Authenticating user token {{(pid=87917) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:38.012083 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.fernet_utils [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87917) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:38.035237 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44815d-1c37-4e75-b94f-b114817ad66b admin admin] Validating token access rules against request {{(pid=87917) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:38.059430 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:38.102876 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd873d67-8de2-46cc-b4c4-9d6f067134df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3709ff21e04749f7aae9073ca845e875', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ed368272c55a45f5a7959b08d8add92c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87917) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:38.103510 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd873d67-8de2-46cc-b4c4-9d6f067134df admin admin] REQUEST_METHOD: `DELETE` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:38.103811 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd873d67-8de2-46cc-b4c4-9d6f067134df admin admin] SCRIPT_NAME: `/identity` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:38.104077 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd873d67-8de2-46cc-b4c4-9d6f067134df admin admin] PATH_INFO: `/v3/projects/c242bca3eda94d1c9ae0121553a0f12f` {{(pid=87917) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:38.105181 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd873d67-8de2-46cc-b4c4-9d6f067134df admin admin] RBAC: Authorizing `identity:delete_project(project_id=c242bca3eda94d1c9ae0121553a0f12f)` {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:38.107101 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd873d67-8de2-46cc-b4c4-9d6f067134df admin admin] RBAC: Authorization granted {{(pid=87917) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:38.156379 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bd873d67-8de2-46cc-b4c4-9d6f067134df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c242bca3eda94d1c9ae0121553a0f12f', 'request_id': 'req-bd873d67-8de2-46cc-b4c4-9d6f067134df'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:38.162791 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bd873d67-8de2-46cc-b4c4-9d6f067134df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c242bca3eda94d1c9ae0121553a0f12f', 'request_id': 'req-bd873d67-8de2-46cc-b4c4-9d6f067134df'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:38.170944 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bd873d67-8de2-46cc-b4c4-9d6f067134df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c242bca3eda94d1c9ae0121553a0f12f', 'request_id': 'req-bd873d67-8de2-46cc-b4c4-9d6f067134df'} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:38.181837 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bd873d67-8de2-46cc-b4c4-9d6f067134df admin admin] The token cache is being invalidate because project c242bca3eda94d1c9ae0121553a0f12f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87917) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:07:38.182126 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bd873d67-8de2-46cc-b4c4-9d6f067134df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:38.182754 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG keystone.notifications [None req-bd873d67-8de2-46cc-b4c4-9d6f067134df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87917) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:38.184232 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: [pid: 87917|app: 0|req: 1477/2938] 10.222.0.55 () {58 vars in 1233 bytes} [Thu Sep 8 14:07:38 2022] DELETE /identity/v3/projects/c242bca3eda94d1c9ae0121553a0f12f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:47.831913 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:48.178127 dv-cinder-820027-24-iscsi devstack@keystone.service[87917]: DEBUG dbcounter [-] [87917] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87917) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:45.882297 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] Authenticating user token {{(pid=87916) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:45.883799 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:46.018072 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777aecb-a285-4896-a8a2-32925b3f1c62 admin admin] Validating token access rules against request {{(pid=87916) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:46.075030 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20788a0f-3bfc-4bb0-9638-298a94236209 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e42fb05dbae64c758098c591d6ffa790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0bbe9c4e7bc14e70b57bf834cfff365f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87916) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:46.075634 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20788a0f-3bfc-4bb0-9638-298a94236209 service placement] REQUEST_METHOD: `GET` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:46.075912 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20788a0f-3bfc-4bb0-9638-298a94236209 service placement] SCRIPT_NAME: `/identity` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:46.076183 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20788a0f-3bfc-4bb0-9638-298a94236209 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87916) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:46.077904 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.fernet_utils [None req-20788a0f-3bfc-4bb0-9638-298a94236209 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87916) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:46.088884 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20788a0f-3bfc-4bb0-9638-298a94236209 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:46.090482 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20788a0f-3bfc-4bb0-9638-298a94236209 service placement] RBAC: Authorization granted {{(pid=87916) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:46.100372 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: [pid: 87916|app: 0|req: 1462/2939] 10.222.0.55 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:56.096154 dv-cinder-820027-24-iscsi devstack@keystone.service[87916]: DEBUG dbcounter [-] [87916] Writing DB stats keystone:SELECT=89 {{(pid=87916) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}